www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DIP10005: Dependency-Carrying Declarations is now available for

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Destroy.

https://github.com/dlang/DIPs/pull/51/files


Andrei
Dec 13 2016
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
(eh I was off by one zero in the title)
Dec 13 2016
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/13/2016 05:33 PM, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


The "View" mode offers formatting before merging: 
https://github.com/andralex/DIPs/blob/10ca2e22c0d759c81e4b3afbdc085d7131f6
b85/DIPs/DIP1005.md 
-- Andrei
Dec 13 2016
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 13.12.2016 23:33, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


1. The syntax is ambiguous.

Is import.foo.bar.baz the symbol baz in module foo.bar, or the symbol 
bar.baz in module foo?

I'd prefer syntax like (import foo.bar).baz and (import foo).bar.baz. 
(I.e., the syntax of import expressions would closely mirror that of 
import declarations, and would be unambiguous.)

2. The behaviour of aliases to import expressions should be defined 
explicitly.

I.e.

alias short = import very.long.module_name;

void foo(int i)(short.T a){ ... }

does this import the module if foo is not instantiated?
Dec 13 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/13/16 6:03 PM, Timon Gehr wrote:

 On 13.12.2016 23:33, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


 1. The syntax is ambiguous.

 Is import.foo.bar.baz the symbol baz in module foo.bar, or the symbol
 bar.baz in module foo?

 I'd prefer syntax like (import foo.bar).baz and (import foo).bar.baz.
 (I.e., the syntax of import expressions would closely mirror that of
 import declarations, and would be unambiguous.)


That is a problem. I switched to the most attractive alternate syntax. 
https://github.com/andralex/DIPs/blob/2e5859c0f64ac4949123fe8de39ccf2ccf72d859/DIPs/DIP1005.md


Andrei
Dec 13 2016
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 14.12.2016 01:14, Andrei Alexandrescu wrote:

 On 12/13/16 6:03 PM, Timon Gehr wrote:

 On 13.12.2016 23:33, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


 1. The syntax is ambiguous.

 Is import.foo.bar.baz the symbol baz in module foo.bar, or the symbol
 bar.baz in module foo?

 I'd prefer syntax like (import foo.bar).baz and (import foo).bar.baz.
 (I.e., the syntax of import expressions would closely mirror that of
 import declarations, and would be unambiguous.)


 That is a problem. I switched to the most attractive alternate syntax.
 https://github.com/andralex/DIPs/blob/2e5859c0f64ac4949123fe8de39ccf2ccf72d859/DIPs/DIP1005.md



 Andrei


I'd still advise to match the syntax of other import declarations. What 
justifies the difference? (Note that import(foo.bar.baz) is a string 
import.)
Dec 13 2016
parent Yuxuan Shui <yshuiv7 gmail.com> writes:
On Wednesday, 14 December 2016 at 00:21:23 UTC, Timon Gehr wrote:

 On 14.12.2016 01:14, Andrei Alexandrescu wrote:

 On 12/13/16 6:03 PM, Timon Gehr wrote:

 [...]


 That is a problem. I switched to the most attractive alternate 
 syntax.
 https://github.com/andralex/DIPs/blob/2e5859c0f64ac4949123fe8de39ccf2ccf72d859/DIPs/DIP1005.md



 Andrei


 I'd still advise to match the syntax of other import 
 declarations. What justifies the difference? (Note that 
 import(foo.bar.baz) is a string import.)


Maybe it's a good idea to make it a property?

e.g.:

 import("std.stdio") void process(File f);
Dec 13 2016
prev sibling parent reply Chris M. <chrismohrfeld comcast.net> writes:
On Tuesday, 13 December 2016 at 23:03:39 UTC, Timon Gehr wrote:

 On 13.12.2016 23:33, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


 1. The syntax is ambiguous.

 Is import.foo.bar.baz the symbol baz in module foo.bar, or the 
 symbol bar.baz in module foo?

 I'd prefer syntax like (import foo.bar).baz and (import 
 foo).bar.baz. (I.e., the syntax of import expressions would 
 closely mirror that of import declarations, and would be 
 unambiguous.)


Why not something like import foo.bar : baz or import foo : 
bar.baz to distinguish between module and symbol? It's already 
used anyway.

Also I like Yuxuan's idea, the other ideas seem to add more 
clutter after the function name
Dec 13 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/13/16 8:07 PM, Chris M. wrote:

 Why not something like import foo.bar : baz or import foo : bar.baz to
 distinguish between module and symbol? It's already used anyway.

 Also I like Yuxuan's idea, the other ideas seem to add more clutter
 after the function name


So we have:

// 1
struct Buffer(R) if ((import std.range).isInputRange!R) { ... }

// 2
struct Buffer(R) if (import std.range:isInputRange!R) { ... }

// 3
 import("std.stdio") struct Buffer(R) if (isInputRange!R) { ... }

The first two require repeating the import for each separate 
declaration, e.g.:

// 1
bool equal(R1, R2)
if ((import std.range).isInputRange!R1 && (import 
std.range).isInputRange!R2)
{ ... }

// 2
bool equal(R1, R2)
if (import std.range:isInputRange!R1 && import std.range:isInputRange!R2)
{ ... }

The last is surprising because it uses a string where otherwise there 
would be an unquoted list of dot-separated names.

I prefer the current form of the proposal:

bool equal(R1, R2)
import (std.range)
if (isInputRange!R1 && isInputRange!R2)
{ ... }

The point has been brought up that the syntax import(std.range) is also 
used for string imports. It is a drawback.


Andrei
Dec 13 2016
next sibling parent reply Chris M. <chrismohrfeld comcast.net> writes:
On Wednesday, 14 December 2016 at 01:39:01 UTC, Andrei 
Alexandrescu wrote:


 I prefer the current form of the proposal:

 bool equal(R1, R2)
 import (std.range)
 if (isInputRange!R1 && isInputRange!R2)
 { ... }

 The point has been brought up that the syntax import(std.range) 
 is also used for string imports. It is a drawback.


 Andrei


How about using "imports" instead of "import"? Simple enough 
change, and it still makes sense

bool equal(R1, R2)
imports (std.range)
if (isInputRange!R1 && isInputRange!R2)
{ ... }
Dec 13 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/13/16 8:53 PM, Chris M. wrote:

 On Wednesday, 14 December 2016 at 01:39:01 UTC, Andrei Alexandrescu wrote:


 I prefer the current form of the proposal:

 bool equal(R1, R2)
 import (std.range)
 if (isInputRange!R1 && isInputRange!R2)
 { ... }

 The point has been brought up that the syntax import(std.range) is
 also used for string imports. It is a drawback.


 Andrei


 How about using "imports" instead of "import"? Simple enough change, and
 it still makes sense

 bool equal(R1, R2)
 imports (std.range)
 if (isInputRange!R1 && isInputRange!R2)
 { ... }


I fear that (a) a lot of folks will believe "imports" is a keyword; (b) 
another lot of folks will write "import" instead of "imports" and wonder 
why it doesn't work. -- Andrei
Dec 13 2016
parent Chris M. <chrismohrfeld comcast.net> writes:
On Wednesday, 14 December 2016 at 02:23:07 UTC, Andrei 
Alexandrescu wrote:

 On 12/13/16 8:53 PM, Chris M. wrote:

 On Wednesday, 14 December 2016 at 01:39:01 UTC, Andrei 
 Alexandrescu wrote:


 I prefer the current form of the proposal:

 bool equal(R1, R2)
 import (std.range)
 if (isInputRange!R1 && isInputRange!R2)
 { ... }

 The point has been brought up that the syntax 
 import(std.range) is
 also used for string imports. It is a drawback.


 Andrei


 How about using "imports" instead of "import"? Simple enough 
 change, and
 it still makes sense

 bool equal(R1, R2)
 imports (std.range)
 if (isInputRange!R1 && isInputRange!R2)
 { ... }


 I fear that (a) a lot of folks will believe "imports" is a 
 keyword; (b) another lot of folks will write "import" instead 
 of "imports" and wonder why it doesn't work. -- Andrei


Could use something like "using" or "with" instead of "import" 
then.

bool equal(R1, R2)
using(std.range)
if (isInputRange!R1 && isInputRange!R2)
{ ... }

Of course, the collision with the syntax for string imports might 
not be a huge deal anyway, since you could easily distinguish 
between when it's for a string import and when it's for a inline 
import just by the context you see it in. Leaving it as "import" 
would probably be fine too.
Dec 13 2016
prev sibling parent reply pineapple <meapineapple gmail.com> writes:
On Wednesday, 14 December 2016 at 01:53:44 UTC, Chris M. wrote:

 How about using "imports" instead of "import"? Simple enough 
 change, and it still makes sense

 bool equal(R1, R2)
 imports (std.range)
 if (isInputRange!R1 && isInputRange!R2)
 { ... }


On Tuesday, 13 December 2016 at 23:03:39 UTC, Timon Gehr wrote:

 I'd prefer syntax like (import foo.bar).baz and (import 
 foo).bar.baz. (I.e., the syntax of import expressions would 
 closely mirror that of import declarations, and would be 
 unambiguous.)

 2. The behaviour of aliases to import expressions should be 
 defined explicitly.

 I.e.

 alias short = import very.long.module_name;

 void foo(int i)(short.T a){ ... }

 does this import the module if foo is not instantiated?


I am most in favor of making the function signature either 
`imports` or ` imports`, or doing this:

struct Buffer(R) if (import std.range:isInputRange!R) { ... }

I also think actually being able to write `alias short = import 
very.long.module_name;` would be great, if only so that the 
contents of a module can be imported into their own user-defined 
namespace rather than the global scope.
Dec 14 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/16 5:55 AM, pineapple wrote:

 On Wednesday, 14 December 2016 at 01:53:44 UTC, Chris M. wrote:

 How about using "imports" instead of "import"? Simple enough change,
 and it still makes sense

 bool equal(R1, R2)
 imports (std.range)
 if (isInputRange!R1 && isInputRange!R2)
 { ... }


 On Tuesday, 13 December 2016 at 23:03:39 UTC, Timon Gehr wrote:

 I'd prefer syntax like (import foo.bar).baz and (import foo).bar.baz.
 (I.e., the syntax of import expressions would closely mirror that of
 import declarations, and would be unambiguous.)

 2. The behaviour of aliases to import expressions should be defined
 explicitly.

 I.e.

 alias short = import very.long.module_name;

 void foo(int i)(short.T a){ ... }

 does this import the module if foo is not instantiated?


 I am most in favor of making the function signature either `imports` or
 ` imports`, or doing this:


No inflections please. Walter requires the use of the "import" keyword, 
and I agree with him.


 struct Buffer(R) if (import std.range:isInputRange!R) { ... }

 I also think actually being able to write `alias short = import
 very.long.module_name;` would be great, if only so that the contents of
 a module can be imported into their own user-defined namespace rather
 than the global scope.


The alias fails to meet the DCD principle: the declaration carries 
dependencies with it.


Andrei
Dec 14 2016
prev sibling next sibling parent reply Hatem Oraby <hatemoraby gmail.com> writes:
On Wednesday, 14 December 2016 at 01:39:01 UTC, Andrei 
Alexandrescu wrote:

 On 12/13/16 8:07 PM, Chris M. wrote:

 [...]


 So we have:

 // 1
 struct Buffer(R) if ((import std.range).isInputRange!R) { ... }

 [...]


What about?:

with(import std.range)
bool equal(R1, R2) if (isInputRange!R1 && isInputRange!R2)
{ ... }
Dec 13 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/13/16 9:22 PM, Hatem Oraby wrote:

 On Wednesday, 14 December 2016 at 01:39:01 UTC, Andrei Alexandrescu wrote:

 On 12/13/16 8:07 PM, Chris M. wrote:

 [...]


 So we have:

 // 1
 struct Buffer(R) if ((import std.range).isInputRange!R) { ... }

 [...]


 What about?:

 with(import std.range)
 bool equal(R1, R2) if (isInputRange!R1 && isInputRange!R2)
 { ... }


I considered this, then figured with is superfluous. -- Andrei
Dec 13 2016
parent reply Jacob Carlborg <doob me.com> writes:
On 2016-12-14 03:23, Andrei Alexandrescu wrote:

 On 12/13/16 9:22 PM, Hatem Oraby wrote:



 with(import std.range)
 bool equal(R1, R2) if (isInputRange!R1 && isInputRange!R2)
 { ... }


 I considered this, then figured with is superfluous. -- Andrei


It could allow to have a better control of the scope which the import 
affects, i.e.:

with(import std.range)
{
   void foo(T) if (isInputRange!T)
   void bar(T) if (isInputRange!T)
}

-- 
/Jacob Carlborg
Dec 13 2016
next sibling parent reply Suliman <evermind live.ru> writes:
On Wednesday, 14 December 2016 at 07:17:57 UTC, Jacob Carlborg 
wrote:

 On 2016-12-14 03:23, Andrei Alexandrescu wrote:

 On 12/13/16 9:22 PM, Hatem Oraby wrote:



 with(import std.range)
 bool equal(R1, R2) if (isInputRange!R1 && isInputRange!R2)
 { ... }


 I considered this, then figured with is superfluous. -- Andrei


 It could allow to have a better control of the scope which the 
 import affects, i.e.:

 with(import std.range)
 {
   void foo(T) if (isInputRange!T)
   void bar(T) if (isInputRange!T)
 }


Look very nice!
Dec 14 2016
parent reply Yuxuan Shui <yshuiv7 gmail.com> writes:
On Wednesday, 14 December 2016 at 08:16:56 UTC, Suliman wrote:

 On Wednesday, 14 December 2016 at 07:17:57 UTC, Jacob Carlborg 
 wrote:

 On 2016-12-14 03:23, Andrei Alexandrescu wrote:

 On 12/13/16 9:22 PM, Hatem Oraby wrote:



 with(import std.range)
 bool equal(R1, R2) if (isInputRange!R1 && isInputRange!R2)
 { ... }


 I considered this, then figured with is superfluous. -- Andrei


 It could allow to have a better control of the scope which the 
 import affects, i.e.:

 with(import std.range)
 {
   void foo(T) if (isInputRange!T)
   void bar(T) if (isInputRange!T)
 }


 Look very nice!


If this looks nice to you then I would like to propose:

 import("std.range"):
void foo(T) if (isInputRange!T)

And:

 import("std.range")
{
   void foo(T) if (isInputRange!T)
   void bar(T) if (isInputRange!T)
}

In my eye this whole thing fits really well into the property 
syntax. Yes, it's a little strange to pass module name as string. 
Otherwise it's perfect.

Maybe we can allow properties to take symbol as argument?
Dec 14 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/16 3:43 AM, Yuxuan Shui wrote:

  import("std.range"):


This seems to break the requirement that declaration-local imports 
should offer the same amenities as regular imports. It would seem much 
if we started putting all of the import syntax in strings. -- Andrei
Dec 14 2016
prev sibling next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 14 December 2016 at 07:17:57 UTC, Jacob Carlborg 
wrote:

 On 2016-12-14 03:23, Andrei Alexandrescu wrote:

 On 12/13/16 9:22 PM, Hatem Oraby wrote:



 with(import std.range)
 bool equal(R1, R2) if (isInputRange!R1 && isInputRange!R2)
 { ... }


 I considered this, then figured with is superfluous. -- Andrei


 It could allow to have a better control of the scope which the 
 import affects, i.e.:

 with(import std.range)
 {
   void foo(T) if (isInputRange!T)
   void bar(T) if (isInputRange!T)
 }


Trouble is, there's no real difference between doing that, vs. 
creating a standalone module containing `foo` and `bar` with 
`import std.range;` as a top-level import.

The `with` syntax could also be used internally in functions to 
narrow the functionality that has access to the imported symbols, 
but something like:

void foo(T)(T input)
{
     with (import some_module)
     {
         // only stuff in here has access
         // to some_module's symbols
     }

     // nothing here has access to
     // some_module's symbols
}

... doesn't give you anything that you can't already get using:

void foo(T)(T input)
{
     {
         import some_module;
         // only stuff in here has access
         // to some_module's symbols
     }

     // nothing here has access to
     // some_module's symbols
}

In fact that's a good argument against the `with` syntax 
entirely, because in typical usage AFAIR, it creates a scope, 
whereas one wouldn't want the symbols defined in Hatem's proposal 
to be scoped relative to the module.

It's a shame, because unlike Andrei I don't feel use of `with` is 
in principle superfluous: it has a value in clarifying intention, 
while allowing the use of traditional `import something` syntax.  
But special-casing of where `with` does and doesn't create a 
scope seems ... less good.
Dec 14 2016
next sibling parent reply Mathias Lang <mathias.lang sociomantic.com> writes:
On Wednesday, 14 December 2016 at 09:01:30 UTC, Joseph Rushton 
Wakeling wrote:

 On Wednesday, 14 December 2016 at 07:17:57 UTC, Jacob Carlborg 
 wrote:

 On 2016-12-14 03:23, Andrei Alexandrescu wrote:

 On 12/13/16 9:22 PM, Hatem Oraby wrote:



 with(import std.range)
 bool equal(R1, R2) if (isInputRange!R1 && isInputRange!R2)
 { ... }


 I considered this, then figured with is superfluous. -- Andrei


 It could allow to have a better control of the scope which the 
 import affects, i.e.:

 with(import std.range)
 {
   void foo(T) if (isInputRange!T)
   void bar(T) if (isInputRange!T)
 }


 Trouble is, there's no real difference between doing that, vs. 
 creating a standalone module containing `foo` and `bar` with 
 `import std.range;` as a top-level import.



That was my impression when reading this DIP. I'm very glad to 
see that decoupling made its way up in the growing list of things 
to do, my only concern is that this syntax sounds like a 
workaround for giant modules.

Phobos is cited as a motivation for this enhancement. Dare I say 
that we have a problem of modules in phobos being too monolithic, 
and they should be split into more packages, like std.range and 
std.algorithms did ?
Dec 14 2016
next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 14 December 2016 at 12:01:40 UTC, Mathias Lang 
wrote:

 That was my impression when reading this DIP. I'm very glad to 
 see that decoupling made its way up in the growing list of 
 things to do, my only concern is that this syntax sounds like a 
 workaround for giant modules.

 Phobos is cited as a motivation for this enhancement. Dare I 
 say that we have a problem of modules in phobos being too 
 monolithic, and they should be split into more packages, like 
 std.range and std.algorithms did ?


Yea, I think you put your finger on it: almost all of the stuff 
this feature could help achieve in Phobos could be just as well 
achieved by splitting stuff up better.

Note, `std.range` and `std.algorithm` could still be much more 
modularized than they currently are.
Dec 14 2016
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Wednesday, 14 December 2016 at 12:15:20 UTC, Joseph Rushton 
Wakeling wrote:

 Yea, I think you put your finger on it: almost all of the stuff 
 this feature could help achieve in Phobos could be just as well 
 achieved by splitting stuff up better.


no, it can't. why should i remember that i have to import 
"std.range.xyz" for something? of course, i will write `import 
std.range;` and forget about it. no kind of splitting will allow 
me to have self-contained "hygienic" declarations.

finally, some DIP that addresses something i really think about 
almost every day.
Dec 14 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/16 7:30 AM, ketmar wrote:

 finally, some DIP that addresses something i really think about almost
 every day.


Additional motivating examples are welcome. -- Andrei
Dec 14 2016
parent ketmar <ketmar ketmar.no-ip.org> writes:
On Wednesday, 14 December 2016 at 14:19:43 UTC, Andrei 
Alexandrescu wrote:

 On 12/14/16 7:30 AM, ketmar wrote:

 finally, some DIP that addresses something i really think 
 about almost
 every day.


 Additional motivating examples are welcome. -- Andrei


it happened that DIP itself has all the samples i can think out. 
;-) imports for function arguments and template constraints are 
the things i want. and i don't really have any favor for any 
particular syntax.

thanks for making this DIP.
Dec 14 2016
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, 14 December 2016 at 12:30:33 UTC, ketmar wrote:

 On Wednesday, 14 December 2016 at 12:15:20 UTC, Joseph Rushton 
 Wakeling wrote:

 Yea, I think you put your finger on it: almost all of the 
 stuff this feature could help achieve in Phobos could be just 
 as well achieved by splitting stuff up better.


 no, it can't. why should i remember that i have to import 
 "std.range.xyz" for something? of course, i will write `import 
 std.range;` and forget about it. no kind of splitting will 
 allow me to have self-contained "hygienic" declarations.


Yeah, modularity from the standpoint of importing a set of 
functions and modularity from the standpoint of what those 
functions import are orthogonal concerns. Without this DIP, we're 
often unable to treat them orthogonally, but in principle, they 
are. Are this DIP (regardless of the exact syntax) would allow us 
to.

- Jonathan M Davis
Dec 14 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/16 7:01 AM, Mathias Lang wrote:

 Trouble is, there's no real difference between doing that, vs.
 creating a standalone module containing `foo` and `bar` with `import
 std.range;` as a top-level import.


 That was my impression when reading this DIP. I'm very glad to see that
 decoupling made its way up in the growing list of things to do, my only
 concern is that this syntax sounds like a workaround for giant modules.

 Phobos is cited as a motivation for this enhancement. Dare I say that we
 have a problem of modules in phobos being too monolithic, and they
 should be split into more packages, like std.range and std.algorithms did ?


Creating many files indeed works around the problem that dependency 
granularity is file-level. I don't think that approach is desirable or 
scalable. -- Andrei
Dec 14 2016
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 14.12.2016 10:01, Joseph Rushton Wakeling wrote:

 On Wednesday, 14 December 2016 at 07:17:57 UTC, Jacob Carlborg wrote:

 On 2016-12-14 03:23, Andrei Alexandrescu wrote:

 On 12/13/16 9:22 PM, Hatem Oraby wrote:



 with(import std.range)
 bool equal(R1, R2) if (isInputRange!R1 && isInputRange!R2)
 { ... }


 I considered this, then figured with is superfluous. -- Andrei


 It could allow to have a better control of the scope which the import
 affects, i.e.:

 with(import std.range)
 {
   void foo(T) if (isInputRange!T)
   void bar(T) if (isInputRange!T)
 }


 Trouble is, there's no real difference between doing that, vs. creating
 a standalone module containing `foo` and `bar` with `import std.range;`
 as a top-level import.
 ...


This point is independent of syntax.



 ...
 It's a shame, because unlike Andrei I don't feel use of `with` is in
 principle superfluous: it has a value in clarifying intention, while
 allowing the use of traditional `import something` syntax.  But
 special-casing of where `with` does and doesn't create a scope seems ...
 less good.


'static with'?
Dec 14 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/16 2:17 AM, Jacob Carlborg wrote:

 On 2016-12-14 03:23, Andrei Alexandrescu wrote:

 On 12/13/16 9:22 PM, Hatem Oraby wrote:



 with(import std.range)
 bool equal(R1, R2) if (isInputRange!R1 && isInputRange!R2)
 { ... }


 I considered this, then figured with is superfluous. -- Andrei


 It could allow to have a better control of the scope which the import
 affects, i.e.:

 with(import std.range)
 {
   void foo(T) if (isInputRange!T)
   void bar(T) if (isInputRange!T)
 }


Thanks for this suggestion.

* "with" is superfluous

* The grouping is not indicating that the import is effected only if one 
of the names is looked up

* I predict 90% of the time there will be one import per declaration, 
which takes it down to:

with (import std.range)
void foo(T) if (isInputRange!T) {}

which underlines the redundancy of the "with".


Andrei
Dec 14 2016
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, 14 December 2016 at 14:11:47 UTC, Andrei 
Alexandrescu wrote:

 * I predict 90% of the time there will be one import per 
 declaration, which takes it down to:

 with (import std.range)
 void foo(T) if (isInputRange!T) {}

 which underlines the redundancy of the "with".


Honestly, while I can see some value in what Jacob is suggesting, 
because it allows you to restrict some imports without repeating 
them, I think that the reality of the matter is that in practice, 
it would result in the imports just being at the top of the 
module - particularly when you consider that function signtaures 
are actually pretty far apart in most cases, since you have the 
function body (which is often not small), the unittest block, and 
the documentation for the next function, making it pretty ugly to 
wrap all of those in an import block.

- Jonathan M Davis
Dec 14 2016
prev sibling parent reply Jon Degenhardt <noreply noreply.com> writes:
On Wednesday, 14 December 2016 at 01:39:01 UTC, Andrei 
Alexandrescu wrote:

 I prefer the current form of the proposal:

 bool equal(R1, R2)
 import (std.range)
 if (isInputRange!R1 && isInputRange!R2)
 { ... }


I didn't see an example of the syntax when multiple imports are 
involved (apologies if I missed it). Would be good to spell it 
out. For example, the start to uninitializedArray (array.d):

     auto uninitializedArray(T, I...)(I sizes) nothrow  system
     if (isDynamicArray!T && allSatisfy!(isIntegral, I) && 
hasIndirections!(ElementEncodingType!T))
     {

This uses imports from std.traits, std.meta, and std.range. Would 
there be one import statement or three? I'm guessing so the 
intent is one, so perhaps:

     auto uninitializedArray(T, I...)(I sizes) nothrow  system
     import (std.traits; std.meta; std.range)
     if (isDynamicArray!T && allSatisfy!(isIntegral, I) && 
hasIndirections!(ElementEncodingType!T))
     {

Is that the idea? Similarly for importing individual symbols. In 
the above case:

     auto uninitializedArray(T, I...)(I sizes) nothrow  system
     import (std.traits : hasIndirections, isDynamicArray, 
isIntegral;
             std.meta : allSatisfy;
             std.range : ElementEncodingType)
     if (isDynamicArray!T && allSatisfy!(isIntegral, I) && 
hasIndirections!(ElementEncodingType!T))
     {

--Jon
Dec 13 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/13/16 10:37 PM, Jon Degenhardt wrote:

 I didn't see an example of the syntax when multiple imports are involved
 (apologies if I missed it). Would be good to spell it out.


Thanks for the point and examples, will change the DIP accordingly. -- 
Andrei
Dec 14 2016
prev sibling next sibling parent reply Suliman <evermind live.ru> writes:
On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu 
wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


Imho such syntaxis construction make language harder to learn. D 
is already pretty complex, but DIPs should simplify language, but 
do not make its harder
Dec 13 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/16 12:00 AM, Suliman wrote:

 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Imho such syntaxis construction make language harder to learn. D is
 already pretty complex, but DIPs should simplify language, but do not
 make its harder


Thanks for the input. My take on this is DIPs should make the language 
_better_. A lot of the time simpler is indeed better, but not always.

The motivation section should cover why we ought to take the hit of a 
complication in the language. Please let me know where it seems lacking.


Andrei
Dec 14 2016
prev sibling next sibling parent reply Dicebot <public dicebot.lv> writes:
On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu 
wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


Proposed functionality does feel useful but syntax options 
mentioned in the DIP all seem lacking to me. Most importantly, 
the one suggested as primary option puts import after the symbol 
in lexical order which looks baroque and may even be 
unprecedented in the language (can't think of another case like 
that immediately).
Dec 14 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/16 3:17 AM, Dicebot wrote:

 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


 Proposed functionality does feel useful but syntax options mentioned in
 the DIP all seem lacking to me. Most importantly, the one suggested as
 primary option puts import after the symbol in lexical order which looks
 baroque and may even be unprecedented in the language (can't think of
 another case like that immediately).


I don't worry about that too much. We have a precedent with names used 
in the return type of a function, and nobody has ever complained about that:

CommonType!(T1, T2) fun(T1, T2)(T1, T2);


Andrei
Dec 14 2016
prev sibling next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 14/12/2016 11:33 AM, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


Others have brought up similar syntax to this:

import(my.mod).Type!argsT(7)

import(Identifier).Identifier

It shouldn't be confused with string imports or the like and would be 
more akin to a selective import.

Pros:
  - Is an expression
  - Relatively clear and in line with string imports as an expression
Cons:
  - Another language feature to learn (but if I was designing a new 
language, this would totally be in it)
  - More text that goes into template constraints
  - Personally I find if I need an import like this in one constraint, I 
want it in a lot more places
  - Much longer lines!

    ```D
void func(T)(T x)
if (import(std.traits).isBasicType!T || 
import(std.traits).isFloatingPoint!T) {

}
    ```
Dec 14 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/16 6:06 AM, rikki cattermole wrote:

 On 14/12/2016 11:33 AM, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


 Others have brought up similar syntax to this:

 import(my.mod).Type!argsT(7)

 import(Identifier).Identifier

 It shouldn't be confused with string imports or the like and would be
 more akin to a selective import.

 Pros:
  - Is an expression


How does it then apply to declarations? -- Andrei
Dec 14 2016
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 15/12/2016 3:16 AM, Andrei Alexandrescu wrote:

 On 12/14/16 6:06 AM, rikki cattermole wrote:

 On 14/12/2016 11:33 AM, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


 Others have brought up similar syntax to this:

 import(my.mod).Type!argsT(7)

 import(Identifier).Identifier

 It shouldn't be confused with string imports or the like and would be
 more akin to a selective import.

 Pros:
  - Is an expression


 How does it then apply to declarations? -- Andrei


Like this?

import(my.mod).Type!argsT value;

Or?

import(my.mod).Type!argsT func(import(my.mod).Type!argsT arg) {

}
Dec 14 2016
prev sibling next sibling parent reply Dominikus Dittes Scherkl <Dominikus.Scherkl continental-corporation.com> writes:
On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu 
wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


Why not leave it as it is and only change the compiler to
perform inputs _within_ a function before evaluating the 
declaration, so that the symbols imported can be used in the 
declaration?

e.g.

fun(Range x) if(isInputRange!x)
{
    import std.range;
    auto a = x.front();
}

this is especially more clean if a function needs several imports 
and maybe some of them in the declaration but not all of them. 
This would otherwise split the imports to two different points, 
some within the (already much too long) declaration and some 
within the function. I consider this ugly.
Dec 14 2016
next sibling parent drug <drug2004 bk.ru> writes:
14.12.2016 16:26, Dominikus Dittes Scherkl пишет:

 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Why not leave it as it is and only change the compiler to
 perform inputs _within_ a function before evaluating the declaration, so
 that the symbols imported can be used in the declaration?

 e.g.

 fun(Range x) if(isInputRange!x)
 {
    import std.range;
    auto a = x.front();
 }

 this is especially more clean if a function needs several imports and
 maybe some of them in the declaration but not all of them. This would
 otherwise split the imports to two different points, some within the
 (already much too long) declaration and some within the function. I
 consider this ugly.


I think this is really smart way to solve the problem.
Dec 14 2016
prev sibling next sibling parent Robert burner Schadek <rburners gmail.com> writes:
I really like that solution. +1
Dec 14 2016
prev sibling next sibling parent Robert burner Schadek via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 2016-12-14 14:26, Dominikus Dittes Scherkl via Digitalmars-d wrote:

 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Why not leave it as it is and only change the compiler to
 perform inputs _within_ a function before evaluating the declaration,
 so that the symbols imported can be used in the declaration?

 e.g.

 fun(Range x) if(isInputRange!x)
 {
    import std.range;
    auto a = x.front();
 }

 this is especially more clean if a function needs several imports and
 maybe some of them in the declaration but not all of them. This would
 otherwise split the imports to two different points, some within the
 (already much too long) declaration and some within the function. I
 consider this ugly.


I like that, +1
Dec 14 2016
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/16 8:26 AM, Dominikus Dittes Scherkl wrote:

 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Why not leave it as it is and only change the compiler to
 perform inputs _within_ a function before evaluating the declaration, so
 that the symbols imported can be used in the declaration?


Thanks. I considered this but it put pressure on the relationship 
between the name and where it's looked up. Consider that imports in a 
function don't need to be at the top. They may also be in nested scopes 
inside the function. It becomes quite tenuous to explain where the 
parameter type names are looked up. -- Andrei
Dec 14 2016
parent reply default0 <Kevin.Labschek gmx.de> writes:
On Wednesday, 14 December 2016 at 14:21:55 UTC, Andrei 
Alexandrescu wrote:

 On 12/14/16 8:26 AM, Dominikus Dittes Scherkl wrote:

 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei 
 Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Why not leave it as it is and only change the compiler to
 perform inputs _within_ a function before evaluating the 
 declaration, so
 that the symbols imported can be used in the declaration?


 Thanks. I considered this but it put pressure on the 
 relationship between the name and where it's looked up. 
 Consider that imports in a function don't need to be at the 
 top. They may also be in nested scopes inside the function. It 
 becomes quite tenuous to explain where the parameter type names 
 are looked up. -- Andrei


Could restrict to imports that come before any other statements 
in the function body, ie:

// valid, compiles when instantiated
void fun(Range)(Range r) if(isInputRange!Range) {
     import std.range;
}

// invalid, import is not picked up because it is not the first 
statement in the method body
void fun(Range)(Range r) if(isInputRange!Range) {
     doThings();
     import std.range;
}

If this is desirable, the error message for failed symbol lookup 
of symbols in a function declaration should hint that imports in 
the function body are only considered if they stand at the 
beginning of the function, to help mitigate confusion.
Seems simple enough to specify and explain, honestly.
Dec 14 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 11:02 AM, default0 wrote:

 On Wednesday, 14 December 2016 at 14:21:55 UTC, Andrei Alexandrescu wrote:

 On 12/14/16 8:26 AM, Dominikus Dittes Scherkl wrote:

 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Why not leave it as it is and only change the compiler to
 perform inputs _within_ a function before evaluating the declaration, so
 that the symbols imported can be used in the declaration?


 Thanks. I considered this but it put pressure on the relationship
 between the name and where it's looked up. Consider that imports in a
 function don't need to be at the top. They may also be in nested
 scopes inside the function. It becomes quite tenuous to explain where
 the parameter type names are looked up. -- Andrei


 Could restrict to imports that come before any other statements in the
 function body, ie:

 // valid, compiles when instantiated
 void fun(Range)(Range r) if(isInputRange!Range) {
     import std.range;
 }

 // invalid, import is not picked up because it is not the first
 statement in the method body
 void fun(Range)(Range r) if(isInputRange!Range) {
     doThings();
     import std.range;
 }

 If this is desirable, the error message for failed symbol lookup of
 symbols in a function declaration should hint that imports in the
 function body are only considered if they stand at the beginning of the
 function, to help mitigate confusion.
 Seems simple enough to specify and explain, honestly.


To me it seems like adding additional rules to make up for a weakness 
that should be avoided in the first place. -- Andrei
Dec 14 2016
parent reply David Gileadi <gileadis NSPMgmail.com> writes:
On 12/14/16 9:27 AM, Andrei Alexandrescu wrote:

 On 12/14/2016 11:02 AM, default0 wrote:

 On Wednesday, 14 December 2016 at 14:21:55 UTC, Andrei Alexandrescu
 wrote:

 On 12/14/16 8:26 AM, Dominikus Dittes Scherkl wrote:

 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu
 wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Why not leave it as it is and only change the compiler to
 perform inputs _within_ a function before evaluating the
 declaration, so
 that the symbols imported can be used in the declaration?


 Thanks. I considered this but it put pressure on the relationship
 between the name and where it's looked up. Consider that imports in a
 function don't need to be at the top. They may also be in nested
 scopes inside the function. It becomes quite tenuous to explain where
 the parameter type names are looked up. -- Andrei


 Could restrict to imports that come before any other statements in the
 function body, ie:

 // valid, compiles when instantiated
 void fun(Range)(Range r) if(isInputRange!Range) {
     import std.range;
 }

 // invalid, import is not picked up because it is not the first
 statement in the method body
 void fun(Range)(Range r) if(isInputRange!Range) {
     doThings();
     import std.range;
 }

 If this is desirable, the error message for failed symbol lookup of
 symbols in a function declaration should hint that imports in the
 function body are only considered if they stand at the beginning of the
 function, to help mitigate confusion.
 Seems simple enough to specify and explain, honestly.


 To me it seems like adding additional rules to make up for a weakness
 that should be avoided in the first place. -- Andrei


For the little that it's worth, my opinion is that it's easier to 
understand/teach the proposed rule above than the new syntax in DIP1005. 
This is true for me whether the proposed rule applies to all imports 
within the top scope of the template function/class/struct, or only to 
imports at the beginning of that scope.

The above rule doesn't cover non-template function declarations like the 
`process` example in the DIP, however. Are they an important enough use 
case to justify new syntax?
Dec 14 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 12:07 PM, David Gileadi wrote:

 The above rule doesn't cover non-template function declarations like the
 `process` example in the DIP, however. Are they an important enough use
 case to justify new syntax?


I suspect 90% of all uses will be straight definitions of template 
functions or template structs/classes. So by that estimate we should be 
in good shape.

However, looking inside the definition in order to look up names in its 
declarations breaks the rule of least astonishment. Making the import 
part of the syntactical unit of the declaration seems to be the path of 
least resistance.


Andrei
Dec 14 2016
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, 14 December 2016 at 17:28:57 UTC, Andrei 
Alexandrescu wrote:

 On 12/14/2016 12:07 PM, David Gileadi wrote:

 The above rule doesn't cover non-template function 
 declarations like the
 `process` example in the DIP, however. Are they an important 
 enough use
 case to justify new syntax?


 I suspect 90% of all uses will be straight definitions of 
 template functions or template structs/classes. So by that 
 estimate we should be in good shape.

 However, looking inside the definition in order to look up 
 names in its declarations breaks the rule of least 
 astonishment. Making the import part of the syntactical unit of 
 the declaration seems to be the path of least resistance.


It also doesn't work with function prototypes. With the proposed 
syntax, you can do

     int foo(SysTime st) import(std.datetime);

but if the compiler has to look into the function body to get at 
the import, then a prototype like this would be out of luck.

- Jonathan M Davis
Dec 14 2016
parent reply Dominikus Dittes Scherkl <Dominikus.Scherkl continental-corporation.com> writes:
[about leaving the syntax as it is, only extend the import to 
include the previous (declaration) line]

On Wednesday, 14 December 2016 at 21:21:39 UTC, Jonathan M Davis 
wrote:

 It also doesn't work with function prototypes. With the 
 proposed syntax, you can do

     int foo(SysTime st) import(std.datetime);

 but if the compiler has to look into the function body to get 
 at the import, then a prototype like this would be out of luck.


You are right. I think this is the only shortcoming.
But how does that work with imports within the function anyway?
In this case you can't even today determine the dependencies of a 
function from it's prototype.

Also I hate prototypes: .di-files are pretty useless - they make 
not clear what is needed to include and for templates you need 
the whole code anyway. I think we don't loose much if now we have 
even symbols within a .di-file which are not declared there.
The only good thing about prototypes is: they can be 
auto-generated. We only need to modify this generation by adding 
all function-local includes to the top of the .di-file.
This would also improve the usefulness of such files by having a 
whole list of dependencies of a library - but this has nothing to 
do with this change! This is necessary independently, because 
local imports are very common today.
Dec 14 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/15/2016 02:23 AM, Dominikus Dittes Scherkl wrote:

 Also I hate prototypes: .di-files are pretty useless - they make not
 clear what is needed to include and for templates you need the whole
 code anyway.


BTW are there outstanding issues with .di files? I recall some solid 
improvements were made a while ago. -- Andrei
Dec 19 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/14/2016 5:26 AM, Dominikus Dittes Scherkl wrote:

 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Why not leave it as it is and only change the compiler to
 perform inputs _within_ a function before evaluating the declaration, so that
 the symbols imported can be used in the declaration?

 e.g.

 fun(Range x) if(isInputRange!x)
 {
    import std.range;
    auto a = x.front();
 }


That would make it problematic to have function declarations.
Dec 15 2016
parent reply Dominikus Dittes Scherkl <Dominikus.Scherkl continental-corporation.com> writes:
On Thursday, 15 December 2016 at 16:16:51 UTC, Walter Bright 
wrote:

 On 12/14/2016 5:26 AM, Dominikus Dittes Scherkl wrote:

 Why not leave it as it is and only change the compiler to
 perform inputs _within_ a function before evaluating the 
 declaration, so that
 the symbols imported can be used in the declaration?

 e.g.

 fun(Range x) if(isInputRange!x)
 {
    import std.range;
    auto a = x.front();
 }


 That would make it problematic to have function declarations.


Yeah, but declarations ARE already problematic, because local 
imports are not visible from them, so only having a declaration 
file (.di) at the moment already is not enough to determine the 
dependencies.
And as I understand you, you don't plan to remove the local 
imports as they are now? (would be a really huge breaking change!)

So, I no longer propose to change nothing except the internal 
compiler behaviour.
Now I propose to additionally change the .di-file generation to 
also add all local imports to the start of the declaration file, 
so that having this file gives ALL dependencies of the declared 
stuff.
Dec 15 2016
next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Thursday, 15 December 2016 at 17:07:35 UTC, Dominikus Dittes 
Scherkl wrote:

 On Thursday, 15 December 2016 at 16:16:51 UTC, Walter Bright 
 wrote:

 On 12/14/2016 5:26 AM, Dominikus Dittes Scherkl wrote:

 Why not leave it as it is and only change the compiler to
 perform inputs _within_ a function before evaluating the 
 declaration, so that
 the symbols imported can be used in the declaration?

 e.g.

 fun(Range x) if(isInputRange!x)
 {
    import std.range;
    auto a = x.front();
 }


 That would make it problematic to have function declarations.


 So, I no longer propose to change nothing except the internal 
 compiler behaviour.
 Now I propose to additionally change the .di-file generation to 
 also add all local imports to the start of the declaration 
 file, so that having this file gives ALL dependencies of the 
 declared stuff.


This makes me think about the current .di generation. As far as I 
remember it only strips function bodies and leaves everything 
else intact. It could already try to remove unused imports. And 
in the future it could replace regular imports with 
auto-generated DCD-style imports.
Dec 15 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/15/2016 01:06 PM, Andrej Mitrovic wrote:

 On Thursday, 15 December 2016 at 17:07:35 UTC, Dominikus Dittes Scherkl
 wrote:

 On Thursday, 15 December 2016 at 16:16:51 UTC, Walter Bright wrote:

 On 12/14/2016 5:26 AM, Dominikus Dittes Scherkl wrote:

 Why not leave it as it is and only change the compiler to
 perform inputs _within_ a function before evaluating the
 declaration, so that
 the symbols imported can be used in the declaration?

 e.g.

 fun(Range x) if(isInputRange!x)
 {
    import std.range;
    auto a = x.front();
 }


 That would make it problematic to have function declarations.


 So, I no longer propose to change nothing except the internal compiler
 behaviour.
 Now I propose to additionally change the .di-file generation to also
 add all local imports to the start of the declaration file, so that
 having this file gives ALL dependencies of the declared stuff.


 This makes me think about the current .di generation. As far as I
 remember it only strips function bodies and leaves everything else
 intact. It could already try to remove unused imports. And in the future
 it could replace regular imports with auto-generated DCD-style imports.


Please make this an issue. Thanks! -- Andrei
Dec 15 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/15/2016 9:07 AM, Dominikus Dittes Scherkl wrote:

 So, I no longer propose to change nothing except the internal compiler
behaviour.
 Now I propose to additionally change the .di-file generation to also add all
 local imports to the start of the declaration file, so that having this file
 gives ALL dependencies of the declared stuff.


That will subtly change the behavior of overloads across module lookups.
Dec 15 2016
parent Dominikus Dittes Scherkl <Dominikus.Scherkl continental-corporation.com> writes:
On Friday, 16 December 2016 at 07:15:51 UTC, Walter Bright wrote:

 On 12/15/2016 9:07 AM, Dominikus Dittes Scherkl wrote:

 So, I no longer propose to change nothing except the internal 
 compiler behaviour.
 Now I propose to additionally change the .di-file generation 
 to also add all
 local imports to the start of the declaration file, so that 
 having this file
 gives ALL dependencies of the declared stuff.


 That will subtly change the behavior of overloads across module 
 lookups.


Hm. Maybe if the .di-file would be used for code generation. But 
does that make sense? A .di-file is delivered together with an 
already compiled library to provide the sources of templates AND 
to show up all dependencies of the library, no? Both for the 
interested reader and the compiler. And the actual .di-files 
doesn't do that correct.
Dec 16 2016
prev sibling next sibling parent Iakh <iaktakh gmail.com> writes:
On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu 
wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


How about something like Haskells "where" to allow any set of 
pre-declaration just for one declaration:

with
{
import ...;
alias Result = ...;
}
Result func(blah) {...}
Dec 14 2016
prev sibling next sibling parent Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu 
wrote:

 Destroy.


FWIW, the reason why I don't use inline imports in my code:

http://d.puremagic.com/issues/show_bug.cgi?id=7016
Dec 14 2016
prev sibling next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu 
wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


How about:

bool equal(R1, R2) : std.range
if (isInputRange!R1 && isInputRange!R2)
{ ... }

It's nice and concise, and you could in theory also allow 
multiple imports with a comma

bool equal(R1, R2) : std.range, std.traits
if (isInputRange!R1 && isInputRange!R2 && isArray!R2)
{ ... }
Dec 14 2016
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Wednesday, 14 December 2016 at 17:01:50 UTC, Andrej Mitrovic 
wrote:

 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei 
 Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


 How about:

 bool equal(R1, R2) : std.range
 if (isInputRange!R1 && isInputRange!R2)
 { ... }

 It's nice and concise, and you could in theory also allow 
 multiple imports with a comma

 bool equal(R1, R2) : std.range, std.traits
 if (isInputRange!R1 && isInputRange!R2 && isArray!R2)
 { ... }


breaks possible selective import. if both modules exports some 
symbol, we will need to selectively import and/or rename it.
Dec 14 2016
next sibling parent reply Anonymouse <asdf asdf.net> writes:
On Wednesday, 14 December 2016 at 17:09:44 UTC, ketmar wrote:

 On Wednesday, 14 December 2016 at 17:01:50 UTC, Andrej Mitrovic

 How about:

 bool equal(R1, R2) : std.range
 if (isInputRange!R1 && isInputRange!R2)
 { ... }

 It's nice and concise, and you could in theory also allow 
 multiple imports with a comma

 bool equal(R1, R2) : std.range, std.traits
 if (isInputRange!R1 && isInputRange!R2 && isArray!R2)
 { ... }


 breaks possible selective import. if both modules exports some 
 symbol, we will need to selectively import and/or rename it.


The with keyword then?

bool equal(R1, R2)
with (std.range : isInputRange, isOutputRange) && (std.stdio : 
writeln) && (std.algorithm)
if (isInputRange!R1 && isInputRange!R2)
{ ... }

A problem with that is that it reads "bool equal with std.stdio 
if something", suggesting something decides whether or not to 
import std.stdio.
Dec 14 2016
next sibling parent reply Anonymouse <asdf asdf.net> writes:
On Wednesday, 14 December 2016 at 17:24:29 UTC, Anonymouse wrote:

 The with keyword then?

 bool equal(R1, R2)
 with (std.range : isInputRange, isOutputRange) && (std.stdio : 
 writeln) && (std.algorithm)
 if (isInputRange!R1 && isInputRange!R2)
 { ... }


I guess the &&s there don't make sense. Maybe with a semicolon 
delimeter instead.

bool equal(R1, R2)
with (std.range : isInputRange, isOutputRange; std.stdio : 
writeln; std.algorithm)
if (isInputRange!R1 && isInputRange!R2)
{ ... }
Dec 14 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 12:28 PM, Anonymouse wrote:

 On Wednesday, 14 December 2016 at 17:24:29 UTC, Anonymouse wrote:

 The with keyword then?

 bool equal(R1, R2)
 with (std.range : isInputRange, isOutputRange) && (std.stdio :
 writeln) && (std.algorithm)
 if (isInputRange!R1 && isInputRange!R2)
 { ... }


 I guess the &&s there don't make sense. Maybe with a semicolon delimeter
 instead.

 bool equal(R1, R2)
 with (std.range : isInputRange, isOutputRange; std.stdio : writeln;
 std.algorithm)
 if (isInputRange!R1 && isInputRange!R2)
 { ... }


This would not pass either. -- Andrei
Dec 14 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 12:24 PM, Anonymouse wrote:

 On Wednesday, 14 December 2016 at 17:09:44 UTC, ketmar wrote:

 On Wednesday, 14 December 2016 at 17:01:50 UTC, Andrej Mitrovic

 How about:

 bool equal(R1, R2) : std.range
 if (isInputRange!R1 && isInputRange!R2)
 { ... }

 It's nice and concise, and you could in theory also allow multiple
 imports with a comma

 bool equal(R1, R2) : std.range, std.traits
 if (isInputRange!R1 && isInputRange!R2 && isArray!R2)
 { ... }


 breaks possible selective import. if both modules exports some symbol,
 we will need to selectively import and/or rename it.


 The with keyword then?

 bool equal(R1, R2)
 with (std.range : isInputRange, isOutputRange) && (std.stdio : writeln)
 && (std.algorithm)
 if (isInputRange!R1 && isInputRange!R2)
 { ... }


The "import" keyword must be used. -- Andrei
Dec 14 2016
prev sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Wednesday, 14 December 2016 at 17:09:44 UTC, ketmar wrote:

 bool equal(R1, R2) : std.range, std.traits
 if (isInputRange!R1 && isInputRange!R2 && isArray!R2)
 { ... }


 breaks possible selective import. if both modules exports some 
 symbol, we will need to selectively import and/or rename it.


Not necessarily. The first colon is the only special case, 
afterwards you can use all forms of imports:


 bool equal(R1, R2) : std.range : isInputRange, Trait = 
 std.traits
 if (isInputRange!R1 && isInputRange!R2 && Trait.isArray!R2)
 { ... }
Dec 14 2016
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Dec 14, 2016 at 05:24:42PM +0000, Andrej Mitrovic via Digitalmars-d
wrote:

 On Wednesday, 14 December 2016 at 17:09:44 UTC, ketmar wrote:

 bool equal(R1, R2) : std.range, std.traits
 if (isInputRange!R1 && isInputRange!R2 && isArray!R2)
 { ... }


 
 breaks possible selective import. if both modules exports some symbol,
 we will need to selectively import and/or rename it.


 
 Not necessarily. The first colon is the only special case, afterwards you
 can use all forms of imports:
 

 bool equal(R1, R2) : std.range : isInputRange, Trait = std.traits
 if (isInputRange!R1 && isInputRange!R2 && Trait.isArray!R2)
 { ... }






What about:

	/* Showing full declaration just for context */
	bool myFunc(R1, R2)(R1 r1, R2 r2)
	import {
		std.range : isInputRange,
		std.traits : isNum = isNumeric
	}
	if (isInputRange!R1 && isInputRange!R2 &&
	    isNum!(ElementType!R1))
	in { assert(someCondition!R1); }
	out(result) { assert(result == expectedResult(...)); }
	body
	{
		...
	}


T

-- 
EMACS = Extremely Massive And Cumbersome System
Dec 14 2016
next sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Wednesday, 14 December 2016 at 17:32:10 UTC, H. S. Teoh wrote:

 What about:

 	/* Showing full declaration just for context */
 	bool myFunc(R1, R2)(R1 r1, R2 r2)
 	import {
 		std.range : isInputRange,
 		std.traits : isNum = isNumeric
 	}
 	if (isInputRange!R1 && isInputRange!R2 &&
 	    isNum!(ElementType!R1))
 	in { assert(someCondition!R1); }
 	out(result) { assert(result == expectedResult(...)); }
 	body
 	{
 		...
 	}


 T


you won't stop me trying to write some code in import section 
then. hey, there are curly brackets, so it SHOULD accept code 
there! ;-)
Dec 14 2016
parent reply Timothee Cour via Digitalmars-d <digitalmars-d puremagic.com> writes:
What about simply this:


```
module foo;

{
import std.stdio;
void fun(File foo){}
}

{
import sd.range;
void foo(T) if(isInputRange!T){}
}

```



On Wed, Dec 14, 2016 at 9:50 AM, ketmar via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Wednesday, 14 December 2016 at 17:32:10 UTC, H. S. Teoh wrote:


 What about:

         /* Showing full declaration just for context */
         bool myFunc(R1, R2)(R1 r1, R2 r2)
         import {
                 std.range : isInputRange,
                 std.traits : isNum = isNumeric
         }
         if (isInputRange!R1 && isInputRange!R2 &&
             isNum!(ElementType!R1))
         in { assert(someCondition!R1); }
         out(result) { assert(result == expectedResult(...)); }
         body
         {
                 ...
         }


 T


 you won't stop me trying to write some code in import section then. hey,
 there are curly brackets, so it SHOULD accept code there! ;-)
Dec 14 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 02:22 PM, Timothee Cour via Digitalmars-d wrote:

 What about simply this:


 ```
 module foo;

 {
 import std.stdio;
 void fun(File foo){}
 }

 {
 import sd.range;
 void foo(T) if(isInputRange!T){}
 }

 ```


Walter proposed this as well. We found two problems with it: (a) it is 
not clear that the import is effected only if the name is looked up; (b) 
the common case is one import clause for one declaration, which makes 
the code awkward. Either you have an indirection level to start with:

{
     import std.range;
     void foo(T) if(isInputRange!T)
     {
         ... one extra indent already ...
     }
}

or the horror of C++ namespaces:

{import std.range;

void foo(T) if(isInputRange!T)
{
    ...
}
}

The fact that C++ namespaces introduce scopes has been an unpleasant 
mistake. Virtually all coding standards I've seen make 
namespace-introduced scopes special in that they don't participate in 
the usual indentation rules. Even special emacs and vi rules have been 
devised for that. Something we can learn from.


Andrei
Dec 14 2016
prev sibling next sibling parent reply Meta <jared771 gmail.com> writes:
On Wednesday, 14 December 2016 at 17:32:10 UTC, H. S. Teoh wrote:

 What about:

 	/* Showing full declaration just for context */
 	bool myFunc(R1, R2)(R1 r1, R2 r2)
 	import {
 		std.range : isInputRange,
 		std.traits : isNum = isNumeric
 	}
 	if (isInputRange!R1 && isInputRange!R2 &&
 	    isNum!(ElementType!R1))
 	in { assert(someCondition!R1); }
 	out(result) { assert(result == expectedResult(...)); }
 	body
 	{
 		...
 	}


 T


The fact that declarations like this WILL show up in Phobos or 
elsewhere if this proposal is implemented triggers my gag reflex. 
There's got to be a better way than tacking yet another 
pre-ambulatory clause to the function body.
Dec 14 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 02:04 PM, Meta wrote:

 On Wednesday, 14 December 2016 at 17:32:10 UTC, H. S. Teoh wrote:

 What about:

     /* Showing full declaration just for context */
     bool myFunc(R1, R2)(R1 r1, R2 r2)
     import {
         std.range : isInputRange,
         std.traits : isNum = isNumeric
     }
     if (isInputRange!R1 && isInputRange!R2 &&
         isNum!(ElementType!R1))
     in { assert(someCondition!R1); }
     out(result) { assert(result == expectedResult(...)); }
     body
     {
         ...
     }


 T


 The fact that declarations like this WILL show up in Phobos or elsewhere
 if this proposal is implemented triggers my gag reflex. There's got to
 be a better way than tacking yet another pre-ambulatory clause to the
 function body.


What I see here is a function definition that has:

* parameters and return types
* dependencies on other modules
* the static condition under which arguments are acceptable
* the dynamic condition under which arguments are acceptable
* the guarantee made upon successful completion
* the implementation

... each of which is:

* easily distinguishable
* opt-in


Andrei
Dec 14 2016
next sibling parent reply ArturG <var.spool.mail700 gmail.com> writes:
On Wednesday, 14 December 2016 at 19:39:55 UTC, Andrei 
Alexandrescu wrote:

 On 12/14/2016 02:04 PM, Meta wrote:

 On Wednesday, 14 December 2016 at 17:32:10 UTC, H. S. Teoh 
 wrote:

 What about:

     /* Showing full declaration just for context */
     bool myFunc(R1, R2)(R1 r1, R2 r2)
     import {
         std.range : isInputRange,
         std.traits : isNum = isNumeric
     }
     if (isInputRange!R1 && isInputRange!R2 &&
         isNum!(ElementType!R1))
     in { assert(someCondition!R1); }
     out(result) { assert(result == expectedResult(...)); }
     body
     {
         ...
     }






as ketmar said, would the import block have any restrictions what 
code could be used inside, would this work?

T1 fun(T1, T2)(T1 t1, T2 t2)
import
{
     version(A) { someMod: T1, T2; }
     else { someOtherMod: T1, T2; }

     // static if or any other code?
}
{
     T1 ret = t1 + t2;
     return ret;
}
Dec 14 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 04:47 PM, ArturG wrote:

 On Wednesday, 14 December 2016 at 19:39:55 UTC, Andrei Alexandrescu wrote:

 On 12/14/2016 02:04 PM, Meta wrote:

 On Wednesday, 14 December 2016 at 17:32:10 UTC, H. S. Teoh wrote:

 What about:

     /* Showing full declaration just for context */
     bool myFunc(R1, R2)(R1 r1, R2 r2)
     import {
         std.range : isInputRange,
         std.traits : isNum = isNumeric
     }
     if (isInputRange!R1 && isInputRange!R2 &&
         isNum!(ElementType!R1))
     in { assert(someCondition!R1); }
     out(result) { assert(result == expectedResult(...)); }
     body
     {
         ...
     }






 as ketmar said, would the import block have any restrictions what code
 could be used inside, would this work?

 T1 fun(T1, T2)(T1 t1, T2 t2)
 import
 {
     version(A) { someMod: T1, T2; }
     else { someOtherMod: T1, T2; }

     // static if or any other code?
 }
 {
     T1 ret = t1 + t2;
     return ret;
 }


The acceptability of the proposal decays exponentially with its 
deviation from existing import syntax. -- Andrei
Dec 14 2016
parent reply arturg <var.spool.mail700 gmail.com> writes:
On Wednesday, 14 December 2016 at 22:56:54 UTC, Andrei 
Alexandrescu wrote:

 The acceptability of the proposal decays exponentially with its 
 deviation from existing import syntax. -- Andrei


sorry, i missed the import keyword :/

T1 fun(T1, T2)(T1 t1, T2 t2)
import
{
      version(A)
      { import someMod: T1, T2; }
      else
      { import someOtherMod: T1, T2;}

      import std.stdio;

// this import block would support only import statemens, version 
and static if etc...

}
{
      t2.writeln;  // use symbols imported by the import block
      T1 ret = t1 + t2;
      return ret;
}
Dec 14 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/16 6:46 PM, arturg wrote:

 On Wednesday, 14 December 2016 at 22:56:54 UTC, Andrei Alexandrescu wrote:

 The acceptability of the proposal decays exponentially with its
 deviation from existing import syntax. -- Andrei


 sorry, i missed the import keyword :/

 T1 fun(T1, T2)(T1 t1, T2 t2)
 import
 {
      version(A)
      { import someMod: T1, T2; }
      else
      { import someOtherMod: T1, T2;}

      import std.stdio;

 // this import block would support only import statemens, version and
 static if etc...

 }


I wrote the proposal and I have difficulty understanding what's meant 
here. Imagine us inflicting this upon our users. -- Andrei
Dec 14 2016
prev sibling parent Meta <jared771 gmail.com> writes:
On Wednesday, 14 December 2016 at 19:39:55 UTC, Andrei 
Alexandrescu wrote:

 On 12/14/2016 02:04 PM, Meta wrote:

 On Wednesday, 14 December 2016 at 17:32:10 UTC, H. S. Teoh 
 wrote:

 What about:

     /* Showing full declaration just for context */
     bool myFunc(R1, R2)(R1 r1, R2 r2)
     import {
         std.range : isInputRange,
         std.traits : isNum = isNumeric
     }
     if (isInputRange!R1 && isInputRange!R2 &&
         isNum!(ElementType!R1))
     in { assert(someCondition!R1); }
     out(result) { assert(result == expectedResult(...)); }
     body
     {
         ...
     }


 T


 The fact that declarations like this WILL show up in Phobos or 
 elsewhere
 if this proposal is implemented triggers my gag reflex. 
 There's got to
 be a better way than tacking yet another pre-ambulatory clause 
 to the
 function body.


 What I see here is a function definition that has:

 * parameters and return types
 * dependencies on other modules
 * the static condition under which arguments are acceptable
 * the dynamic condition under which arguments are acceptable
 * the guarantee made upon successful completion
 * the implementation

 ... each of which is:

 * easily distinguishable
 * opt-in


 Andrei


Don't forget "looks awful and cluttered as hell".
Dec 14 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 12:32 PM, H. S. Teoh via Digitalmars-d wrote:

 	import {
 		std.range : isInputRange,
 		std.traits : isNum = isNumeric
 	}


This seems ambiguous. In

import { a : b, c.d }

did you meant to import c.d from a, or a different module?


Andrei
Dec 14 2016
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Dec 14, 2016 at 02:30:29PM -0500, Andrei Alexandrescu via Digitalmars-d
wrote:

 On 12/14/2016 12:32 PM, H. S. Teoh via Digitalmars-d wrote:

 	import {
 		std.range : isInputRange,
 		std.traits : isNum = isNumeric
 	}


 
 This seems ambiguous. In
 
 import { a : b, c.d }
 
 did you meant to import c.d from a, or a different module?
 


[...]

That's easy to fix:

	import { a : b, c;  d: e, f; }


T

-- 
There is no gravity. The earth sucks.
Dec 14 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 03:33 PM, H. S. Teoh via Digitalmars-d wrote:

 That's easy to fix:

 	import { a : b, c;  d: e, f; }


The cost being adding new syntax. -- Andrei
Dec 14 2016
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Dec 14, 2016 at 04:08:56PM -0500, Andrei Alexandrescu via Digitalmars-d
wrote:

 On 12/14/2016 03:33 PM, H. S. Teoh via Digitalmars-d wrote:

 That's easy to fix:
 
 	import { a : b, c;  d: e, f; }


 
 The cost being adding new syntax. -- Andrei


Aren't we already adding new syntax?

On thinking about this again, though, isn't the whole point of this DIP
to add a way of attaching imports X, Y, Z to some declaration such that
the imports are in effect for the scope of the declaration?  The
specific syntax is really secondary.  I think we should rather be more
concerned about the semantics of it, such as:

(1) The scope of the imports: I assume the import would be in effect
until the end of the function body, correct? Or does it only last until
the beginning of the function body?

   (a) Are DCD-imported symbols accessible in in- and out-contracts?

   (b) Are DCD imports permitted in the definition of inner functions?
   What is their effect, if allowed? Will symbols thus imported be
   restricted to the body of the inner function, or will it be visible
   throughout the outer function body?

(2) How overloads are handled:

   (a) If the current module declares symbol X, and a DCD import pulls
   in another module that declares X, which X will it resolve to when
   the declaration references X?  Are there cases of possible symbol
   hijacking here?  What if the function body also declares a local
   symbol X?

   (b) If there are multiple DCD imports that declare symbol X, how
   would overload resolution work?

   (c) How are parameter symbols figured into overload resolution? E.g.,
   if the DCD import pulls in module M that declares a module global
   xyz, and one of the function parameter names is also xyz.

   (d) Will symbols in a DCD imported module that aren't specifically
   imported shadow module globals, parameters, or local variables?
   E.g., you import my.module : X, but my.module also declares Y, and
   there's a {module global (in the current module), parameter, local
   variable} called Y.  How would overload resolution work in this case?

(4) Are DCD imports restricted to function declarations, or template
declarations, or all declarations in general?

   (a) I.e., is it possible to declare a module global of type X, where
   X is defined by another module pulled in via a DCD import?
   
   (b) What about the other eponymous template shorthands, like `enum
   E(X) = ...`: will it be possible to attach DCD imports to that? E.g.,
   if we want to say isInputRange!X in the enum definition.

   (c) Is it possible to attach DCD imports to delegate / lambda
   definitions? If so, what is their scope / interaction with the
   containing scope?

(5) Is it possible to attach DCD imports to individual parameters? E.g.:

	auto myFunc(import.std.stdio.File f) { ... }

	or:

	auto myFunc(scope void delegate(import.std.stdio.File f) dg)
	{ ... }

   Or do such imports have to be attached to the top-level declaration?


T

-- 
Computerese Irregular Verb Conjugation: I have preferences.  You have
biases.  He/She has prejudices. -- Gene Wirchenko
Dec 14 2016
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/16 6:11 PM, H. S. Teoh via Digitalmars-d wrote:

 On Wed, Dec 14, 2016 at 04:08:56PM -0500, Andrei Alexandrescu via
Digitalmars-d wrote:

 On 12/14/2016 03:33 PM, H. S. Teoh via Digitalmars-d wrote:

 That's easy to fix:

 	import { a : b, c;  d: e, f; }


 The cost being adding new syntax. -- Andrei


 Aren't we already adding new syntax?


That's not a Boolean. One may be adding a little or a lot.


 On thinking about this again, though, isn't the whole point of this DIP
 to add a way of attaching imports X, Y, Z to some declaration such that
 the imports are in effect for the scope of the declaration?


That is correct.


 The
 specific syntax is really secondary.


That is also correct, and thanks for bringing it up; discussion of 
syntax eclipse other aspects. I'll make a pass to integrate your points.


Andrei
Dec 14 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 06:11 PM, H. S. Teoh via Digitalmars-d wrote:

 On thinking about this again, though, isn't the whole point of this DIP
 to add a way of attaching imports X, Y, Z to some declaration such that
 the imports are in effect for the scope of the declaration?  The
 specific syntax is really secondary.  I think we should rather be more
 concerned about the semantics of it, such as:


[snip]

Thanks very much for your review. I just pushed a commit addressing it. 
-- Andrei
Dec 19 2016
prev sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
On Wednesday, 14 December 2016 at 17:24:42 UTC, Andrej Mitrovic 
wrote:

 On Wednesday, 14 December 2016 at 17:09:44 UTC, ketmar wrote:

 bool equal(R1, R2) : std.range, std.traits
 if (isInputRange!R1 && isInputRange!R2 && isArray!R2)
 { ... }


 breaks possible selective import. if both modules exports some 
 symbol, we will need to selectively import and/or rename it.


 Not necessarily. The first colon is the only special case, 
 afterwards you can use all forms of imports:


 bool equal(R1, R2) : std.range : isInputRange, Trait = 
 std.traits
 if (isInputRange!R1 && isInputRange!R2 && Trait.isArray!R2)
 { ... }






i know that C school likes to add punctuation everywhere ;-), but 
i can't see why that is better than:

bool equal(R1, R2) import(std.range : isInputRange, Trait = 
std.traits) if ...

i think that the version with explicit keyword is easier to read.
Dec 14 2016
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/13/2016 05:33 PM, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


I just realized there is a ddoc consequence to this: ddoc could generate 
the import list and make that available to the generated documentation. 
I'll add that to the DIP. -- Andrei
Dec 14 2016
prev sibling next sibling parent reply bitwise <bitwise.pvt gmail.com> writes:
On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu 
wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


I think the DIP indicated you may have considered this, but why 
not just remove the requirement of 'static import ..' for using 
fully qualified names?

struct Buffered(Range) if (std.range.isInputRange!Range)
{
     import std.range;
     // ...
}

You could then simply qualify any parameters or constraints and 
use local imports for the body.

     Bit
Dec 14 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 03:13 PM, bitwise wrote:

 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


 I think the DIP indicated you may have considered this, but why not just
 remove the requirement of 'static import ..' for using fully qualified
 names?

 struct Buffered(Range) if (std.range.isInputRange!Range)
 {
     import std.range;
     // ...
 }

 You could then simply qualify any parameters or constraints and use
 local imports for the body.


We'd like people and simplistic tools such as grep to indicate quickly 
where imports are present. -- Andrei
Dec 14 2016
parent reply Seb <seb wilzba.ch> writes:
On Wednesday, 14 December 2016 at 20:16:29 UTC, Andrei 
Alexandrescu wrote:

 On 12/14/2016 03:13 PM, bitwise wrote:

 I think the DIP indicated you may have considered this, but 
 why not just
 remove the requirement of 'static import ..' for using fully 
 qualified
 names?

 struct Buffered(Range) if (std.range.isInputRange!Range)
 {
     import std.range;
     // ...
 }

 You could then simply qualify any parameters or constraints 
 and use
 local imports for the body.


 We'd like people and simplistic tools such as grep to indicate 
 quickly where imports are present. -- Andrei


 From all ideas I read so far, this is my favorite. It's a really 
concise syntax.

What is the use case for using "grep" to find all imports? Even 
for something simplistic as to find all unittest blocks, one 
already needs to use Dscanner, so I don't see why this is a 
problem. Just image this case for "grep":

import std.algorithm,
std.range,
std.traits;

Moreover for a list of all imports, there is already a dedicated 
flag in DMD (-deps).
Dec 14 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 09:21 PM, Seb wrote:

 What is the use case for using "grep" to find all imports?


The main idea here is to use "import" for importing things. -- Andrei
Dec 14 2016
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Made a pass through the document integrating a lot of feedback and 
fleshing the proposal better:

https://github.com/andralex/DIPs/blob/155ff59984b26749af7830aeb172d3af2dae8cd7/DIPs/DIP1005.md

https://github.com/dlang/DIPs/pull/51/files


Andrei
Dec 14 2016
next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Wednesday, 14 December 2016 at 22:55:26 UTC, Andrei 
Alexandrescu wrote:

 Made a pass through the document integrating a lot of feedback 
 and fleshing the proposal better:

 https://github.com/andralex/DIPs/blob/155ff59984b26749af7830aeb172d3af2dae8cd7/DIPs/DIP1005.md

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


I'd also mention that the advantage of this feature is for the 
human reader (and not just ddoc generators) as you can instantly 
tell where all the symbols are located which a function uses. It 
could save a lot of time for the reader when analyzing a function.

For example, if we have a function like the following:

ubyte[] readSomeBytes ( )
{
     return read(1024);
}

It's a non-trivial exercise for the reader to understand where 
the `read` symbol is coming from. It usually involves looking up 
the import list at the top of the module which is typically very 
long and contains unrelated imports as the DIP mentions.

Sometimes you can use a symbol lookup shortcut in your editor, 
but with a name like `read` you'll likely get a dozen modules 
listed in the pop-up and you still couldn't immediately tell 
which one of the symbols this `read` call refers to. Someone will 
mention IDEs, but remember when you're looking at the code in 
another environment (say reviewing code on Github) you may not 
have quick point & click functionality to find the symbol 
declaration. That's another win for this DIP.
Dec 14 2016
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 15.12.2016 00:17, Andrej Mitrovic wrote:

 ubyte[] readSomeBytes ( )
 {
     return read(1024);
 }

 It's a non-trivial exercise for the reader to understand where the
 `read` symbol is coming from.


pragma(msg,fullyQualifiedName!read);
Dec 14 2016
next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Wednesday, 14 December 2016 at 23:38:19 UTC, Timon Gehr wrote:

 On 15.12.2016 00:17, Andrej Mitrovic wrote:

 ubyte[] readSomeBytes ( )
 {
     return read(1024);
 }

 It's a non-trivial exercise for the reader to understand where 
 the
 `read` symbol is coming from.


 pragma(msg,fullyQualifiedName!read);


Right, which requires building!
Dec 14 2016
parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Thursday, 15 December 2016 at 00:44:02 UTC, Andrej Mitrovic 
wrote:

 On Wednesday, 14 December 2016 at 23:38:19 UTC, Timon Gehr 
 wrote:

 On 15.12.2016 00:17, Andrej Mitrovic wrote:

 ubyte[] readSomeBytes ( )
 {
     return read(1024);
 }

 It's a non-trivial exercise for the reader to understand 
 where the
 `read` symbol is coming from.


 pragma(msg,fullyQualifiedName!read);


 Right, which requires building!


Also, this wouldn't work with overloads would it? :)
Dec 14 2016
prev sibling parent David Gileadi <gileadis NSPMgmail.com> writes:
On 12/14/16 4:38 PM, Timon Gehr wrote:

 On 15.12.2016 00:17, Andrej Mitrovic wrote:

 ubyte[] readSomeBytes ( )
 {
     return read(1024);
 }

 It's a non-trivial exercise for the reader to understand where the
 `read` symbol is coming from.


 pragma(msg,fullyQualifiedName!read);


Or, for the IDE-addict, hover or control-click on `read`.
Dec 14 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 06:17 PM, Andrej Mitrovic wrote:

 On Wednesday, 14 December 2016 at 22:55:26 UTC, Andrei Alexandrescu wrote:

 Made a pass through the document integrating a lot of feedback and
 fleshing the proposal better:

 https://github.com/andralex/DIPs/blob/155ff59984b26749af7830aeb172d3af2dae8cd7/DIPs/DIP1005.md


 https://github.com/dlang/DIPs/pull/51/files


 Andrei


 I'd also mention that the advantage of this feature is for the human
 reader (and not just ddoc generators) as you can instantly tell where
 all the symbols are located which a function uses. It could save a lot
 of time for the reader when analyzing a function.

 For example, if we have a function like the following:

 ubyte[] readSomeBytes ( )
 {
     return read(1024);
 }

 It's a non-trivial exercise for the reader to understand where the
 `read` symbol is coming from. It usually involves looking up the import
 list at the top of the module which is typically very long and contains
 unrelated imports as the DIP mentions.

 Sometimes you can use a symbol lookup shortcut in your editor, but with
 a name like `read` you'll likely get a dozen modules listed in the
 pop-up and you still couldn't immediately tell which one of the symbols
 this `read` call refers to. Someone will mention IDEs, but remember when
 you're looking at the code in another environment (say reviewing code on
 Github) you may not have quick point & click functionality to find the
 symbol declaration. That's another win for this DIP.


Cool, added that. -- Andrei
Dec 19 2016
prev sibling parent reply Chris M <chrismohrfeld comcast.net> writes:
On Wednesday, 14 December 2016 at 22:55:26 UTC, Andrei 
Alexandrescu wrote:

 Made a pass through the document integrating a lot of feedback 
 and fleshing the proposal better:

 https://github.com/andralex/DIPs/blob/155ff59984b26749af7830aeb172d3af2dae8cd7/DIPs/DIP1005.md

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


For the "with" keyword, why is the import statement kept? Seems 
unnecessary to keep it.

void process(File input) import std.stdio;
struct Buffered(Range) if (isInputRange!Range)
with (std.range)
{
     ...
}
Dec 14 2016
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Thursday, 15 December 2016 at 00:29:17 UTC, Chris M wrote:

 For the "with" keyword, why is the import statement kept? Seems 
 unnecessary to keep it.

 void process(File input) import std.stdio;
 struct Buffered(Range) if (isInputRange!Range)
 with (std.range)
 {
     ...
 }


to not block future uses of such `with` syntax for different 
purposes. and (which is more important for me, for example) to 
make greping easier. and to keep `with` semantics: it should 
introduce scope over *existing* things. explicit `import` brings 
the module to the existance. choose any options you like. ;-)
Dec 14 2016
parent reply Timothee Cour via Digitalmars-d <digitalmars-d puremagic.com> writes:
how about this:

variant 1 // currently legal D; just need to attach semantics

```
// applies to all below
 deps!({import std.stdio; pragma(lib, "curl"); }):

// applies to 1 below
 deps!({import std.range})
void fun(T)(isInputRange!T){} // depends on both deps

void fun2(File file){}  // depends on 1st deps ending with ':'

```

variant 2: //currently legal if not ending with ':' as above; less noisy
syntax
 deps{import std.stdio; pragma(lib, "curl"); }
 deps{import std.stdio; pragma(lib, "curl"); }: // applies to all below

Advantages:

* no new syntax (just new semantics)
* no nesting inside {}
* allows grouping multiple declarations under a single set of imports
* `grep import` will still work unlike many of the existing proposals
* can use existing traits to query for such imports (eg getSymbolsByUDA)
* allows a more general solution (eg also handles `pragma(lib, "curl");` if
we want that as well)




On Wed, Dec 14, 2016 at 4:54 PM, ketmar via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Thursday, 15 December 2016 at 00:29:17 UTC, Chris M wrote:


 For the "with" keyword, why is the import statement kept? Seems
 unnecessary to keep it.

 void process(File input) import std.stdio;
 struct Buffered(Range) if (isInputRange!Range)
 with (std.range)
 {
     ...
 }


 to not block future uses of such `with` syntax for different purposes. and
 (which is more important for me, for example) to make greping easier. and
 to keep `with` semantics: it should introduce scope over *existing* things.
 explicit `import` brings the module to the existance. choose any options
 you like. ;-)
Dec 14 2016
next sibling parent Yuxuan Shui <yshuiv7 gmail.com> writes:
On Thursday, 15 December 2016 at 01:59:33 UTC, Timothee Cour 
wrote:

 how about this:

 variant 1 // currently legal D; just need to attach semantics

 ```
 // applies to all below
  deps!({import std.stdio; pragma(lib, "curl"); }):

 // applies to 1 below
  deps!({import std.range})
 void fun(T)(isInputRange!T){} // depends on both deps

 void fun2(File file){}  // depends on 1st deps ending with ':'

 ```

 variant 2: //currently legal if not ending with ':' as above; 
 less noisy
 syntax
  deps{import std.stdio; pragma(lib, "curl"); }
  deps{import std.stdio; pragma(lib, "curl"); }: // applies to 
 all below

 Advantages:

 * no new syntax (just new semantics)
 * no nesting inside {}
 * allows grouping multiple declarations under a single set of 
 imports
 * `grep import` will still work unlike many of the existing 
 proposals
 * can use existing traits to query for such imports (eg 
 getSymbolsByUDA)
 * allows a more general solution (eg also handles `pragma(lib, 
 "curl");` if
 we want that as well)


Really nice idea! Except it's still a little noisy. I would be 
really happy if this gets accepted...
Dec 14 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 08:59 PM, Timothee Cour via Digitalmars-d wrote:

 how about this:

 variant 1 // currently legal D; just need to attach semantics

 ```
 // applies to all below
  deps!({import std.stdio; pragma(lib, "curl"); }):


Thanks, added. -- Andrei
Dec 19 2016
prev sibling next sibling parent reply Meta <jared771 gmail.com> writes:
On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu 
wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


What's the main goal with this proposal? Is it to introduce DCDs 
into the language or is it to clean up some warts of the local 
import pattern, i.e.:

int doSomething(T)(T t) //std.range is lazily imported but
{                       //can't be used in the signature
     import std.range;
     //...
}

struct Test(R) //Ditto
{
     import std.algorithm;
     //etc...
}

I'm assuming it's the former, or there wouldn't be any talk about 
introducing any new syntax. Imports would just be made completely 
lazy and we wouldn't have to change the language at all 
otherwise. So other than the fact that you can move the 
declaration anywhere without breaking it due to missing imports, 
are the advantages that DCDs provide really worth it enough to 
introduce yet another clause that can be put on any declaration? 
Especially when we have another solution (lazy imports) that does 
exactly what you want but excludes the DCD aspect?
Dec 14 2016
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Thursday, 15 December 2016 at 03:26:24 UTC, Meta wrote:

 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei 
 Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


 What's the main goal with this proposal? Is it to introduce 
 DCDs into the language or is it to clean up some warts of the 
 local import pattern, i.e.:

 int doSomething(T)(T t) //std.range is lazily imported but
 {                       //can't be used in the signature
     import std.range;
     //...
 }

 struct Test(R) //Ditto
 {
     import std.algorithm;
     //etc...
 }

 I'm assuming it's the former, or there wouldn't be any talk 
 about introducing any new syntax. Imports would just be made 
 completely lazy and we wouldn't have to change the language at 
 all otherwise. So other than the fact that you can move the 
 declaration anywhere without breaking it due to missing 
 imports, are the advantages that DCDs provide really worth it 
 enough to introduce yet another clause that can be put on any 
 declaration? Especially when we have another solution (lazy 
 imports) that does exactly what you want but excludes the DCD 
 aspect?


I think there is a case to be made to use the import in the 
symbol's signature, which is currently not possible. Not sure it 
pays for itself to add a feature in the language just for this.

The tooling case is moot IMO. The languages keeps being fubared 
with obtuse complexities, which makes tooling harder to write, 
and then this is used an an excuse to fubar it even more. This 
isn't leading to a good place.

Now there are some ways I see this happen leveraging the 'with' 
creating a "with import" construct.

void foo(R)(R r) if (isInputRange!R) with import std.range {
     // ...
}

The good thing here is that the import syntax doesn't need to be 
altered in any way. You can still do "with import foo.bar : buzz".
Dec 14 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 10:26 PM, Meta wrote:

 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


 What's the main goal with this proposal?


If the document does not clarify that, please let me know of specifics. 
Thanks. -- Andrei
Dec 15 2016
parent reply Meta <jared771 gmail.com> writes:
On Thursday, 15 December 2016 at 13:49:26 UTC, Andrei 
Alexandrescu wrote:

 On 12/14/2016 10:26 PM, Meta wrote:

 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei 
 Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


 What's the main goal with this proposal?


 If the document does not clarify that, please let me know of 
 specifics. Thanks. -- Andrei


Well of course it is about DCDs, but I am talking more about the 
underlying motivation. The listed benefits of DCDs from the DIP:

1. Specifies dependencies at declaration level, not at module 
level. This allows reasoning about the dependency cost of 
declarations in separation instead of aggregated at module level.

2. Dependency-Carrying Declarations are easier to move around, 
making for simpler and faster refactorings.

3. Dependency-Carrying Declarations allow scalable template 
libraries... Distributing a template library in the form of 
Dependency-Carrying Declarations creates a scalable, 
pay-as-you-go setup.

It seems to me that making all imports lazy accomplishes 3, which 
is the real prize. 1 and 2 are nice to have, but it doesn't seem 
to me that the added complexity of the inline import feature is 
worth it when the biggest bang for our buck is not 1 or 2, but 3.

So my question is, what do you want? Is your primary goal to 
introduce DCDs into the language to allow for easier refactoring 
and having it made more clear which imports a declaration uses, 
or is it to make imports "pay as you go" such that a user only 
pays for what they actually use and it's more or less transparent 
to them? You argue in the DIP that DCDs give us all 3 so we don't 
have to choose, but as I said previously, DCDs will not give us 
the biggest ROI compared to the cost of the feature.
Dec 15 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/15/2016 09:31 AM, Meta wrote:

 On Thursday, 15 December 2016 at 13:49:26 UTC, Andrei Alexandrescu wrote:

 On 12/14/2016 10:26 PM, Meta wrote:

 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


 What's the main goal with this proposal?


 If the document does not clarify that, please let me know of
 specifics. Thanks. -- Andrei


 Well of course it is about DCDs, but I am talking more about the
 underlying motivation. The listed benefits of DCDs from the DIP:

 1. Specifies dependencies at declaration level, not at module level.
 This allows reasoning about the dependency cost of declarations in
 separation instead of aggregated at module level.

 2. Dependency-Carrying Declarations are easier to move around, making
 for simpler and faster refactorings.

 3. Dependency-Carrying Declarations allow scalable template libraries...
 Distributing a template library in the form of Dependency-Carrying
 Declarations creates a scalable, pay-as-you-go setup.

 It seems to me that making all imports lazy accomplishes 3, which is the
 real prize. 1 and 2 are nice to have, but it doesn't seem to me that the
 added complexity of the inline import feature is worth it when the
 biggest bang for our buck is not 1 or 2, but 3.

 So my question is, what do you want? Is your primary goal to introduce
 DCDs into the language to allow for easier refactoring and having it
 made more clear which imports a declaration uses, or is it to make
 imports "pay as you go" such that a user only pays for what they
 actually use and it's more or less transparent to them? You argue in the
 DIP that DCDs give us all 3 so we don't have to choose, but as I said
 previously, DCDs will not give us the biggest ROI compared to the cost
 of the feature.


Thanks for sharing your opinion. Mine (and the argument made by DIP1005) 
is that on balance packing dependencies together with declarations is 
worth the language addition. The document does specify the advantages 
and disadvantages of lazy imports, as follows:

===
* Full lazy `import`s. Assume all `import`s are lazy without any change 
in the language. That would allow an idiom by which libraries use fully 
qualified names everywhere, and the `import`s would be effected only 
when names are looked up. This would allow scalability but not the 
dependency-carrying aspect.
===

If that is misrepresenting things, please let me know so I can improve 
the DIP.


Andrei
Dec 15 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/15/2016 6:53 AM, Andrei Alexandrescu wrote:

 The document does specify the advantages and disadvantages of lazy
 imports, as follows:

 ===
 * Full lazy `import`s. Assume all `import`s are lazy without any change in the
 language. That would allow an idiom by which libraries use fully qualified
names
 everywhere, and the `import`s would be effected only when names are looked up.
 This would allow scalability but not the dependency-carrying aspect.
 ===


That would be a massive breaking change.
Dec 15 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/15/2016 11:11 AM, Walter Bright wrote:

 On 12/15/2016 6:53 AM, Andrei Alexandrescu wrote:

 The document does specify the advantages and disadvantages of lazy
 imports, as follows:

 ===
 * Full lazy `import`s. Assume all `import`s are lazy without any
 change in the
 language. That would allow an idiom by which libraries use fully
 qualified names
 everywhere, and the `import`s would be effected only when names are
 looked up.
 This would allow scalability but not the dependency-carrying aspect.
 ===


 That would be a massive breaking change.


This may be a misunderstanding. The idiom would be opt-in so existing 
behavior will be preserved (albeit it won't benefit of the improvements).

Andrei
Dec 15 2016
next sibling parent reply Timothee Cour via Digitalmars-d <digitalmars-d puremagic.com> writes:
Some more details on my proposa based on UDA:

// applies to next decl
 deps!({import std.algorithm;})
void test1(){}

// applies to a set of decls
 deps!({import std.stdio;}){
  void test2(){}
  void test3(){}
}

// applies to all following decls (':')
 deps!({import std.array;}):

// can specify other dependencies beyond imports and have arbitrary complex
logic:
 deps!({
  import std.range;
  static int[100] data2;
  version(linux){
    enum data1=import("foo");//string import
    pragma(lib, "curl");
  }
}):
void test4(){}

// Can alias some dependencies:
alias deps1=deps!({import std.algorithm;});

 deps1
void test4(){}

NOTE: the above code compiles if we add
`struct deps(T...){}`, but that logic would be implemented in the compiler.

Advantages:

* no new syntax (just new semantics)
* no nesting inside {} required, so no additional indentation
* allows grouping multiple declarations under a single set of imports
* `grep import` will still work unlike many of the existing proposals
* can use existing traits to query for such imports (eg getSymbolsByUDA)
* allows a more general solution (eg also handles `pragma(lib, "curl");` if
we want that as well)

(Note: ignore the `variant 2` i had described above)

On Thu, Dec 15, 2016 at 8:32 AM, Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 12/15/2016 11:11 AM, Walter Bright wrote:


 On 12/15/2016 6:53 AM, Andrei Alexandrescu wrote:


 The document does specify the advantages and disadvantages of lazy
 imports, as follows:

 ===
 * Full lazy `import`s. Assume all `import`s are lazy without any
 change in the
 language. That would allow an idiom by which libraries use fully
 qualified names
 everywhere, and the `import`s would be effected only when names are
 looked up.
 This would allow scalability but not the dependency-carrying aspect.
 ===


 That would be a massive breaking change.


 This may be a misunderstanding. The idiom would be opt-in so existing
 behavior will be preserved (albeit it won't benefit of the improvements).

 Andrei
Dec 15 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/15/2016 02:22 PM, Timothee Cour via Digitalmars-d wrote:

 Some more details on my proposa based on UDA:

 // applies to next decl
  deps!({import std.algorithm;})
 void test1(){}

 // applies to a set of decls
  deps!({import std.stdio;}){
   void test2(){}
   void test3(){}
 }

 // applies to all following decls (':')
  deps!({import std.array;}):

 // can specify other dependencies beyond imports and have arbitrary
 complex logic:
  deps!({
   import std.range;
   static int[100] data2;
   version(linux){
     enum data1=import("foo");//string import
     pragma(lib, "curl");
   }
 }):
 void test4(){}

 // Can alias some dependencies:
 alias deps1=deps!({import std.algorithm;});

  deps1
 void test4(){}

 NOTE: the above code compiles if we add
 `struct deps(T...){}`, but that logic would be implemented in the compiler.


I now understand the idea, thank you.

My question is, doesn't this take things too far? Earlier I wrote:


 The acceptability of the proposal decays exponentially with its
 deviation from existing import syntax.


Indeed adding less syntax is better, but that's not an absolute; the 
optimum isn't necessarily at the "zero syntax added" point. This is 
because there are several things to harmonize in language design, which 
naturally are in tension.


Andrei
Dec 15 2016
parent reply ArturG <var.spool.mail700 gmail.com> writes:
On Thursday, 15 December 2016 at 19:52:50 UTC, Andrei 
Alexandrescu wrote:

 On 12/15/2016 02:22 PM, Timothee Cour via Digitalmars-d wrote:

 Some more details on my proposa based on UDA:

 ...


 I now understand the idea, thank you.

 My question is, doesn't this take things too far? Earlier I 
 wrote:


 The acceptability of the proposal decays exponentially with its
 deviation from existing import syntax.


 Indeed adding less syntax is better, but that's not an 
 absolute; the optimum isn't necessarily at the "zero syntax 
 added" point. This is because there are several things to 
 harmonize in language design, which naturally are in tension.


 Andrei


Something like Timothee Cour's  deps proposal is interesting,
because it adds the same options to DCD's as module level imports 
have.

They can be applied to any symbol that supports udas which the 
current syntax from the dip doesnt.

As he displayed, they can be used to group symbols together which 
makes it more dry, but that can also be a drawback when 
refactoring depending which version you use.

And you dont have to support all shown features as that syntax is 
easier extendable.

The initial version could support only import statements.
Dec 15 2016
parent reply Timothee Cour via Digitalmars-d <digitalmars-d puremagic.com> writes:
one more thing:

we can simplify further (while still having formatted looking code) with
!q{} instead of !() :

```
// applies to next decl
 deps!q{import std.algorithm;}
void test1(){}

// applies to a set of decls
 deps!q{import std.stdio;}{
  void test2(){}
  void test3(){}
}

// applies to all following decls (':')
 deps!q{import std.array;}:

// can specify other dependencies beyond imports and have arbitrary complex
logic:
 deps!q{
  import std.range;
  static int[100] data2;
  version(linux){
    enum data1=import("foo");//string import
    pragma(lib, "curl");
  }
}:
void test4(){}

// Can alias some dependencies:
alias deps1=deps!q{import std.algorithm;};

 deps1
void test4(){}
```



On Thu, Dec 15, 2016 at 3:46 PM, ArturG via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Thursday, 15 December 2016 at 19:52:50 UTC, Andrei Alexandrescu wrote:


 On 12/15/2016 02:22 PM, Timothee Cour via Digitalmars-d wrote:


 Some more details on my proposa based on UDA:

 ...

 I now understand the idea, thank you.


 My question is, doesn't this take things too far? Earlier I wrote:

 The acceptability of the proposal decays exponentially with its

 deviation from existing import syntax.


 Indeed adding less syntax is better, but that's not an absolute; the
 optimum isn't necessarily at the "zero syntax added" point. This is because
 there are several things to harmonize in language design, which naturally
 are in tension.


 Andrei


 Something like Timothee Cour's  deps proposal is interesting,
 because it adds the same options to DCD's as module level imports have.

 They can be applied to any symbol that supports udas which the current
 syntax from the dip doesnt.

 As he displayed, they can be used to group symbols together which makes it
 more dry, but that can also be a drawback when refactoring depending which
 version you use.

 And you dont have to support all shown features as that syntax is easier
 extendable.

 The initial version could support only import statements.
Dec 15 2016
parent Yuxuan Shui <yshuiv7 gmail.com> writes:
On Friday, 16 December 2016 at 00:53:14 UTC, Timothee Cour wrote:

 one more thing:

 we can simplify further (while still having formatted looking 
 code) with
 !q{} instead of !() :

 ```
 // applies to next decl
  deps!q{import std.algorithm;}
 void test1(){}

 // applies to a set of decls
  deps!q{import std.stdio;}{
   void test2(){}
   void test3(){}
 }

 // applies to all following decls (':')
  deps!q{import std.array;}:


q{} would suggest the argument is a string, right? Plus, why does 
deps have to be a template?


 // can specify other dependencies beyond imports and have 
 arbitrary complex
 logic:
  deps!q{
   import std.range;
   static int[100] data2;
   version(linux){
     enum data1=import("foo");//string import
     pragma(lib, "curl");
   }
 }:


This is unnecessarily powerful.


 void test4(){}

 // Can alias some dependencies:
 alias deps1=deps!q{import std.algorithm;};

  deps1
 void test4(){}


I would prefer my  import alternative. If we make use of q{}, it 
would look fine too:

 import(q{std.file: File})  import(q{std.range: isInputRange})
void foo(T)(File x) if (isInputRange!T);


 ```



 On Thu, Dec 15, 2016 at 3:46 PM, ArturG via Digitalmars-d < 
 digitalmars-d puremagic.com> wrote:


 [...]
Dec 15 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 12/15/2016 8:32 AM, Andrei Alexandrescu wrote:

 On 12/15/2016 11:11 AM, Walter Bright wrote:

 On 12/15/2016 6:53 AM, Andrei Alexandrescu wrote:

 The document does specify the advantages and disadvantages of lazy
 imports, as follows:

 ===
 * Full lazy `import`s. Assume all `import`s are lazy without any
 change in the
 language. That would allow an idiom by which libraries use fully
 qualified names
 everywhere, and the `import`s would be effected only when names are
 looked up.
 This would allow scalability but not the dependency-carrying aspect.
 ===


 That would be a massive breaking change.


 This may be a misunderstanding. The idiom would be opt-in so existing behavior
 will be preserved (albeit it won't benefit of the improvements).


If existing behavior is preserved, how can it lazily parse the imports?
Dec 15 2016
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 15 December 2016 at 16:11:56 UTC, Walter Bright 
wrote:

 That would be a massive breaking change.


SDC do parse the module only when an identifier resolution reach 
top level, and then populate the module's top level symbol table 
without running any semantic analysis on any of its symbols.

Symbol are analyzed on demand when they are used. I think this 
achieve something close enough to what Andrei was proposing and 
is not breaking.

You may also want to look at 
https://www.youtube.com/watch?v=b_T-eCToX1I to see what clang's 
up to.
Dec 15 2016
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/15/16 4:48 PM, deadalnix wrote:

 On Thursday, 15 December 2016 at 16:11:56 UTC, Walter Bright wrote:

 That would be a massive breaking change.


 SDC do parse the module only when an identifier resolution reach top
 level, and then populate the module's top level symbol table without
 running any semantic analysis on any of its symbols.

 Symbol are analyzed on demand when they are used. I think this achieve
 something close enough to what Andrei was proposing and is not breaking.


Yah, that would work (note we are not talking about breakage; perhaps 
you meant no language addition?). That would (a) miss the dependency 
carrying aspect and (b) would be suboptimal. The moment any unqualified 
symbol is looked up in a module, all modules it imports need to be 
loaded. This could be avoided by using consistently fully qualified symbols.

The comment about alternatives in the DIP does mention that but without 
going into details. Let me know if you think they should be added.


Andrei
Dec 15 2016
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 12/15/2016 1:48 PM, deadalnix wrote:

 On Thursday, 15 December 2016 at 16:11:56 UTC, Walter Bright wrote:

 That would be a massive breaking change.


 SDC do parse the module only when an identifier resolution reach top level, and
 then populate the module's top level symbol table without running any semantic
 analysis on any of its symbols.


I have a hard time seeing in practice where one would ever find it not
necessary 
to parse for symbols. Note that 'object' is implicitly imported, so just using 
'size_t' means every top level import has to be parsed.



 Symbol are analyzed on demand when they are used.


I agree that is not breaking and actually is something I've planned to
implement 
in dmd for a while.
Dec 15 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 12/15/2016 1:48 PM, deadalnix wrote:

 You may also want to look at https://www.youtube.com/watch?v=b_T-eCToX1I to see
 what clang's up to.


Thanks. I see they've done much that Zortech C++ did to be fast, as well as
Warp 
and dmd.

dmd eliminates the need for lib and ar by building in the code for that, so 
object files do not have to be written then read. I've often thought that this 
should be done with the linker, too. I.e. have dmd generate an exe file
directly.
Dec 15 2016
prev sibling next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 12/13/16 11:33 PM, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


On first it seems like an awesome idea. That solves ... but wait what? 
Thinking more  about the problem at hand - I fail to see what this DIP 
accomplishes that can't be done better today.

1. The benefit of placing import to each function is based on the untold 
assumption that we have:
a) huge modules with many functions
b) that constraints of these functions require different (large) modules

The reality is far from this picture - if anything 99% of template 
constraints are dependent on std.range.primitives and std.traits with a 
bit of std.meta from time to time. So we'd have a boilerplate of

auto foo(R)(R range)
(import std.range.primitives)
if(isInputRange!R){ ... }

everywhere for no noticeable benefit - touch one of functions and you 
get full set of imports of these _small_ modules.

2. By itself the mechanism for delaying import even for constraint until 
the function is touched is moot as long as the module in question is 
huge and is not split in pieces. In other words:

auto foo(R)(R range)
(import std.range)
if(isInputRange!R){ ... }

Pulls in full std.range the moment foo is touched, compared to

import std.range.primitives;
...
auto foo(R)(R range)
if(isInputRange!R){ ... }

which is because it actually isolates the whole mess of complete 
std.range from the user of a template.

All in all my practical response is split the modules at least in 2 
parts: constraints + full functionality, then import the one with 
constraints at the top level. Works today and solves the practical 
issues unlike the proposal.

---
Dmitry Olshansky
Dec 15 2016
next sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Thursday, 15 December 2016 at 22:56:42 UTC, Dmitry Olshansky 
wrote:

 On 12/13/16 11:33 PM, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


 On first it seems like an awesome idea. That solves ... but 
 wait what? Thinking more  about the problem at hand - I fail to 
 see what this DIP accomplishes that can't be done better today.

 1. The benefit of placing import to each function is based on 
 the untold assumption that we have:
 a) huge modules with many functions
 b) that constraints of these functions require different 
 (large) modules

 The reality is far from this picture - if anything 99% of 
 template constraints are dependent on std.range.primitives and 
 std.traits with a bit of std.meta from time to time. So we'd 
 have a boilerplate of

 auto foo(R)(R range)
 (import std.range.primitives)
 if(isInputRange!R){ ... }

 everywhere for no noticeable benefit - touch one of functions 
 and you get full set of imports of these _small_ modules.

 2. By itself the mechanism for delaying import even for 
 constraint until the function is touched is moot as long as the 
 module in question is huge and is not split in pieces. In other 
 words:

 auto foo(R)(R range)
 (import std.range)
 if(isInputRange!R){ ... }

 Pulls in full std.range the moment foo is touched, compared to

 import std.range.primitives;
 ...
 auto foo(R)(R range)
 if(isInputRange!R){ ... }

 which is because it actually isolates the whole mess of 
 complete std.range from the user of a template.

 All in all my practical response is split the modules at least 
 in 2 parts: constraints + full functionality, then import the 
 one with constraints at the top level. Works today and solves 
 the practical issues unlike the proposal.

 ---
 Dmitry Olshansky


+1

I understand the motivation for the proposal but I think that 
good practice with package/module organisation renders the 
benefits marginal. The flipside of that is that introducing this 
new syntax reduces the pressure on people to organise their code 
sensibly.

Is it a feature I'd like to be able to use when I'm writing: yes

Is it a feature I actually want to see in D and in the D 
codebases I read daily: probably no, meh

Is it worth the time and effort of core developers and community 
reviewers, both now and as an ongoing feature-maintenance burden: 
absolutely not. I have no idea why this proposal is a priority 
right now.
Dec 16 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/16/2016 04:58 AM, John Colvin wrote:

 Is it worth the time and effort of core developers and community
 reviewers, both now and as an ongoing feature-maintenance burden:
 absolutely not. I have no idea why this proposal is a priority right now.


Walter and I wanted to have a positive example of a good DIP that is 
relatively simple, is noncontroversial, and marks a definite move forward.

We believe the feature's benefits are significant, obvious, and 
immediate. With this feature, any large project would be able to 
radically clarify and improve its dependency structure with only little 
refactoring work. I interpret the negative feedback as failings of the 
DIP to explain and argue matters appropriately, so I'm working hard on 
improving it. Please keep feedback coming.


Thanks,

Andrei
Dec 16 2016
parent reply Jon Degenhardt <noreply noreply.com> writes:
On Friday, 16 December 2016 at 17:19:56 UTC, Andrei Alexandrescu 
wrote:

 Walter and I wanted to have a positive example of a good DIP 
 that is relatively simple, is noncontroversial, and marks a 
 definite move forward.

 We believe the feature's benefits are significant, obvious, and 
 immediate. With this feature, any large project would be able 
 to radically clarify and improve its dependency structure with 
 only little refactoring work. I interpret the negative feedback 
 as failings of the DIP to explain and argue matters 
 appropriately, so I'm working hard on improving it. Please keep 
 feedback coming.


A suggestion: Now that the proposal details are largely fleshed 
out, take a file from phobos and rewrite it in the new style. 
Should give insight into aesthetics and identify remaining issues 
to resolve.

--Jon
Dec 16 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/16/2016 01:24 PM, Jon Degenhardt wrote:

 A suggestion: Now that the proposal details are largely fleshed out,
 take a file from phobos and rewrite it in the new style. Should give
 insight into aesthetics and identify remaining issues to resolve.


What a great idea. For starters, I attempted the simple exercise of 
converting all module-level imports of std.array into static imports. I 
stopped here:

https://github.com/dlang/phobos/pull/4962

Soon the naive approach "let's replace this import with a static import 
and rebuild" ran into the midst of a comedy of errors. Code in various 
other modules fails to compile. Sometimes (happy case) the error 
indicates the symbol that is not seen, which is easy to fix. Other times 
(as you may see if you try to touch the PR above) there is zero 
indication on where the problem originates.

I invite anyone who believes DIP1005 is solving just a minor problem 
and/or that we can rein in by using lazy imports and no improvement to 
the language, to try their hand at continuing the PR above or starting a 
similar effort. Currently it is exhaustingly difficult to figure where 
names are looked up in a large D project.


Andrei
Dec 16 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 12/16/2016 11:59 AM, Andrei Alexandrescu wrote:

 Soon the naive approach "let's replace this import with a static import and
 rebuild" ran into the midst of a comedy of errors. Code in various other
modules
 fails to compile. Sometimes (happy case) the error indicates the symbol that is
 not seen, which is easy to fix. Other times (as you may see if you try to touch
 the PR above) there is zero indication on where the problem originates.


The switch:

     -verrors=spec

is very useful here. It prints the errors otherwise gagged when inside 
__traits(compiles,...).
Dec 17 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/16/2016 01:24 PM, Jon Degenhardt wrote:

 On Friday, 16 December 2016 at 17:19:56 UTC, Andrei Alexandrescu wrote:

 Walter and I wanted to have a positive example of a good DIP that is
 relatively simple, is noncontroversial, and marks a definite move
 forward.

 We believe the feature's benefits are significant, obvious, and
 immediate. With this feature, any large project would be able to
 radically clarify and improve its dependency structure with only
 little refactoring work. I interpret the negative feedback as failings
 of the DIP to explain and argue matters appropriately, so I'm working
 hard on improving it. Please keep feedback coming.


 A suggestion: Now that the proposal details are largely fleshed out,
 take a file from phobos and rewrite it in the new style. Should give
 insight into aesthetics and identify remaining issues to resolve.


I also submit this as evidence:

https://github.com/dlang/phobos/pull/4963

This was a successful attempt to convert all imports to selective 
imports in std.array. It was a very difficult process that took a long 
time, as shown in the comment.


Andrei
Dec 16 2016
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/15/2016 05:56 PM, Dmitry Olshansky wrote:

 On 12/13/16 11:33 PM, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


 On first it seems like an awesome idea. That solves ... but wait what?
 Thinking more  about the problem at hand - I fail to see what this DIP
 accomplishes that can't be done better today.


[snip]

Thanks for this great feedback. I have improved the DIP to integrate it:

Commit: 
https://github.com/dlang/DIPs/pull/51/commits/8f86c5add037c2bfb519dbcd646d1946b0e96571

Current file: https://github.com/dlang/DIPs/pull/51/files

View: 
https://github.com/dlang/DIPs/blob/8f86c5add037c2bfb519dbcd646d1946b0e96571/DIPs/DIP1005.md



Andrei
Dec 16 2016
parent reply Chris Wright <dhasenan gmail.com> writes:
Just looking at this again:


 The obvious workaround to the problem that dependencies must be module-
 level is to simply define many small modules---in the extreme, one per
 declaration.


Andrei works in phobos a lot. Phobos has a lot of large modules. For 
instance, std.datetime is 35,000 lines. It's not unusual for a phobos 
module to have over 6,000 lines (std.math, std.typecons, std.traits, 
std.format, std.conv).

I'd normally recommend breaking up modules at one fifth that size. The 
standard library benefits from low granularity modules. It needs to 
implement a variety of related tools for working with particular things.

For the hunting-for-definitions case, you also need:

* a module with more than a few imports, from different libraries or 
packages
* ambiguous names, or functions that are widely used
* the user can't use an IDE / ctags / dcd
* the user can't use ddox / dpldocs.info, which turns type references 
into links; or the user is using that and needs to find the definition of 
a template constraint
* the maintainer cannot use selective imports
* the maintainer cannot break the module up to reduce the number of 
dependencies
* the maintainer is willing to spend the effort to convert top-level 
imports into tightly scoped imports

For the compilation-speed case, you need:

* large dependencies that this allows you to skip (the module combines 
several types of functionality with different dependencies)
* the imported module must be in another compilation unit (incremental 
compilation or a separate library)
* the dependencies can't be used by any other module in the compilation 
unit
* no selective imports
* the module being compiled depends on something in the same scope

That's a pretty marginal use case.
Dec 17 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/17/2016 02:34 PM, Chris Wright wrote:

 Just looking at this again:


 The obvious workaround to the problem that dependencies must be module-
 level is to simply define many small modules---in the extreme, one per
 declaration.


 Andrei works in phobos a lot. Phobos has a lot of large modules. For
 instance, std.datetime is 35,000 lines. It's not unusual for a phobos
 module to have over 6,000 lines (std.math, std.typecons, std.traits,
 std.format, std.conv).


Let's take a look at that hypothesis. The example I chose randomly (and 
which turned to be a rat's nest of fuzzy dependencies) was std/array.d, 
clocking at 3585 lines. Then looking at the entire project:

wc -l std/*.d 
std/{algorithm,container,digest,experimental,internal,net,range,regex}/**/*.d 
| sort --key=1 -n | cat -n

This outputs the modules in the standard library (excluding those that 
are simple header translations), sorted by LoC, numbered. See result in 
http://paste.ofcode.org/Lc5xfcs8GqpT2cabApSSgk. That shows 137 modules, 
median length 903, average length 2055 --- including full documentation, 
unittests, and examples. These numbers seem quite reasonable and if 
anything compare favorably against other projects I've been on.


 I'd normally recommend breaking up modules at one fifth that size.


Yeah, std/datetime.d is a monster, from what I can tell owing to a rote 
and redundant way of handling unittesting. I didn't look at its 
dependencies, but I doubt they are special. I was quite vocal about 
breaking it up, but I got mellower with time since (a) someone measured 
its size without unittests and it was something like one order of 
magnitude smaller, and (b) there was really no more trouble using or 
maintaining it than with anything else in Phobos.

I should also add that each large project has a couple of outliers like 
that. I even recall a switch of a couple thousand lines once :o).


 The
 standard library benefits from low granularity modules. It needs to
 implement a variety of related tools for working with particular things.

 For the hunting-for-definitions case, you also need:

 * a module with more than a few imports, from different libraries or
 packages
 * ambiguous names, or functions that are widely used
 * the user can't use an IDE / ctags / dcd
 * the user can't use ddox / dpldocs.info, which turns type references
 into links; or the user is using that and needs to find the definition of
 a template constraint
 * the maintainer cannot use selective imports
 * the maintainer cannot break the module up to reduce the number of
 dependencies
 * the maintainer is willing to spend the effort to convert top-level
 imports into tightly scoped imports

 For the compilation-speed case, you need:

 * large dependencies that this allows you to skip (the module combines
 several types of functionality with different dependencies)
 * the imported module must be in another compilation unit (incremental
 compilation or a separate library)
 * the dependencies can't be used by any other module in the compilation
 unit
 * no selective imports
 * the module being compiled depends on something in the same scope

 That's a pretty marginal use case.


Most of these have been the case with all C++ and D projects I've been 
involved with at Facebook.

Please let me know what of this information I should include in the DIP 
to make it better. Thanks.


Andrei
Dec 17 2016
next sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Sat, 17 Dec 2016 19:34:12 -0500, Andrei Alexandrescu wrote:

 Most of these have been the case with all C++ and D projects I've been
 involved with at Facebook.



problem because it encourages larger namespaces), and somewhat with 
Python (though much worse because of the lack of static typing).

D doesn't have either of those pitfalls, so I haven't seen it cause 
problems. I'm also a bit skeptical that this will see much use outside 
phobos.

This isn't really an argument against it. I just don't see any argument 
for it, not that's supported by my own experience.


 Please let me know what of this information I should include in the DIP
 to make it better. Thanks.


You don't mention improved documentation as an option for readability.

As a general comment, the Rationale section takes a while to get to the 
benefits (and with it the associated problems that the DIP is trying to 
address).

My advisor in college exhorted us to be *prefix-competitive* in our 
papers and talks: your audience is present at the start but will get 
bored and wander off at some random point, so make sure that you maximize 
the impact of the part of the message they hear. Every prefix of your 
talk should include as much value as you can pack into that many words.
Dec 17 2016
parent reply pineapple <meapineapple gmail.com> writes:
On Sunday, 18 December 2016 at 02:40:59 UTC, Chris Wright wrote:

 D doesn't have either of those pitfalls, so I haven't seen it 
 cause problems. I'm also a bit skeptical that this will see 
 much use outside phobos.

 This isn't really an argument against it. I just don't see any 
 argument for it, not that's supported by my own experience.


I would like to echo this sentiment.

I am developing a general-use library for D that is currently 
resting at around 50,000 lines. I have never felt a need for a 
feature like this, and I can't imagine a reason to begin using 
it. Dependency management has just never presented an issue. Very 
nearly all modules in the library are fewer than 1,000 lines long 
and very nearly all symbols are selectively imported, and the 
approach has proven to be completely manageable.

If it can be added without interfering with the existing 
patterns, I don't really have an argument against this feature. 
But I do think that what this DIP is meant to address is not 
really a problem experienced by all or even most who are working 
with D. It's a problem being experienced with Phobos, but there 
are very valid solutions to that problem that don't involve an 
addition to the language - only some refactoring. I think that 
makes the argument in favor somewhat weak.
Dec 17 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/17/16 10:21 PM, pineapple wrote:

 On Sunday, 18 December 2016 at 02:40:59 UTC, Chris Wright wrote:

 D doesn't have either of those pitfalls, so I haven't seen it cause
 problems. I'm also a bit skeptical that this will see much use outside
 phobos.

 This isn't really an argument against it. I just don't see any
 argument for it, not that's supported by my own experience.


 I would like to echo this sentiment.

 I am developing a general-use library for D that is currently resting at
 around 50,000 lines.


Is the source code publicly available?


 I have never felt a need for a feature like this,
 and I can't imagine a reason to begin using it. Dependency management
 has just never presented an issue. Very nearly all modules in the
 library are fewer than 1,000 lines long and very nearly all symbols are
 selectively imported, and the approach has proven to be completely
 manageable.

 If it can be added without interfering with the existing patterns, I
 don't really have an argument against this feature. But I do think that
 what this DIP is meant to address is not really a problem experienced by
 all or even most who are working with D.


I recall Liran Zvibel repeatedly mentioned this as a big pain point at 
Weka.io. I will reach out to him.

On what basis do you extrapolate from personal experience to most or all 
who are working with D?


 It's a problem being
 experienced with Phobos, but there are very valid solutions to that
 problem that don't involve an addition to the language - only some
 refactoring. I think that makes the argument in favor somewhat weak.


How is Phobos special?

What kind of refactoring do you envision would improve dependency 
management and build times?


Thanks,

Andrei
Dec 18 2016
parent reply pineapple <meapineapple gmail.com> writes:
On Sunday, 18 December 2016 at 13:31:48 UTC, Andrei Alexandrescu 
wrote:

 On 12/17/16 10:21 PM, pineapple wrote:

 I am developing a general-use library for D that is currently 
 resting at
 around 50,000 lines.


 Is the source code publicly available?


https://github.com/pineapplemachine/mach.d
Dec 18 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/2016 10:01 AM, pineapple wrote:

 On Sunday, 18 December 2016 at 13:31:48 UTC, Andrei Alexandrescu wrote:

 On 12/17/16 10:21 PM, pineapple wrote:

 I am developing a general-use library for D that is currently resting at
 around 50,000 lines.


 Is the source code publicly available?


 https://github.com/pineapplemachine/mach.d


The code looks clean, congrats. With your permission I'd like to give 
this library as an example in the "Workaround: Increasing Granularity of 
Modules" section. Please advise, thanks. -- Andrei
Dec 18 2016
parent reply pineapple <meapineapple gmail.com> writes:
On Sunday, 18 December 2016 at 21:09:46 UTC, Andrei Alexandrescu 
wrote:

 On 12/18/2016 10:01 AM, pineapple wrote:

 On Sunday, 18 December 2016 at 13:31:48 UTC, Andrei 
 Alexandrescu wrote:

 Is the source code publicly available?


 https://github.com/pineapplemachine/mach.d


 The code looks clean, congrats. With your permission I'd like 
 to give this library as an example in the "Workaround: 
 Increasing Granularity of Modules" section. Please advise, 
 thanks. -- Andrei


No problem, feel free. Thank you for asking.
Dec 18 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/2016 05:18 PM, pineapple wrote:

 On Sunday, 18 December 2016 at 21:09:46 UTC, Andrei Alexandrescu wrote:

 On 12/18/2016 10:01 AM, pineapple wrote:

 On Sunday, 18 December 2016 at 13:31:48 UTC, Andrei Alexandrescu wrote:

 Is the source code publicly available?


 https://github.com/pineapplemachine/mach.d


 The code looks clean, congrats. With your permission I'd like to give
 this library as an example in the "Workaround: Increasing Granularity
 of Modules" section. Please advise, thanks. -- Andrei


 No problem, feel free. Thank you for asking.


Is there a simple command to e.g. unittest everything in the project? 
Also, is there a build process or it's all templated? -- Andrei
Dec 18 2016
parent pineapple <meapineapple gmail.com> writes:
On Sunday, 18 December 2016 at 22:31:34 UTC, Andrei Alexandrescu 
wrote:

 Is there a simple command to e.g. unittest everything in the 
 project? Also, is there a build process or it's all templated? 
 -- Andrei


There's no build process. To run tests, I compile the root 
`package.d` file with rdmd, including the -debug, -unittest, and 
--main flags, and -I"path/to/repo/mach.d". To compile the 
mach.sdl package, which is not currently imported by the root 
`package.d`, you would have to include a directory containing the 
requisite Derelict dependencies for the -I option.
Dec 18 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/2016 05:18 PM, pineapple wrote:

 On Sunday, 18 December 2016 at 21:09:46 UTC, Andrei Alexandrescu wrote:

 On 12/18/2016 10:01 AM, pineapple wrote:

 On Sunday, 18 December 2016 at 13:31:48 UTC, Andrei Alexandrescu wrote:

 Is the source code publicly available?


 https://github.com/pineapplemachine/mach.d


 The code looks clean, congrats. With your permission I'd like to give
 this library as an example in the "Workaround: Increasing Granularity
 of Modules" section. Please advise, thanks. -- Andrei


 No problem, feel free. Thank you for asking.


Great, thanks. Please take a look at the accuracy of the discussion. I 
expanded the "Workaround" section and moved it near the top.

https://github.com/dlang/DIPs/pull/51

https://github.com/dlang/DIPs/blob/dd46252e820dce66df746540d7ab94e0b00a6505/DIPs/DIP1005.md


Andrei
Dec 18 2016
next sibling parent pineapple <meapineapple gmail.com> writes:
On Sunday, 18 December 2016 at 23:18:27 UTC, Andrei Alexandrescu 
wrote:

 Great, thanks. Please take a look at the accuracy of the 
 discussion. I expanded the "Workaround" section and moved it 
 near the top.

 https://github.com/dlang/DIPs/pull/51

 https://github.com/dlang/DIPs/blob/dd46252e820dce66df746540d7ab94e0b00a6505/DIPs/DIP1005.md


 Andrei


It's reasonably accurate. The only thing I would point out is 
that few modules define only one public symbol; the median 
probably lies around 4 or 5. A few modules, most or all being in 
the traits package, define considerably more. The 
mach.traits.primitives module defines 28 public symbols, by my 
count. (Though that's still small stuff relative to Phobos.)

I think it would be most accurate to word the sentence as:


 It is organized as such that each module contains a small 
 number of related declarations (such as canAdjoin, Adjoin, and 
 AdjoinFlat in module adjoin.d) along with documentation and 
 unit tests.
Dec 18 2016
prev sibling next sibling parent reply pineapple <meapineapple gmail.com> writes:
On Sunday, 18 December 2016 at 23:18:27 UTC, Andrei Alexandrescu 
wrote:

 Great, thanks. Please take a look at the accuracy of the 
 discussion. I expanded the "Workaround" section and moved it 
 near the top.


I would also like to register that while I respect your argument 
regarding scalability, I have personally found that a greater 
number of smaller files is easier to manage than a smaller number 
of larger files. Including for the 580,000+ line project I work 
on for a living.
Dec 18 2016
next sibling parent pineapple <meapineapple gmail.com> writes:
On Monday, 19 December 2016 at 00:44:14 UTC, pineapple wrote:

 On Sunday, 18 December 2016 at 23:18:27 UTC, Andrei 
 Alexandrescu wrote:

 Great, thanks. Please take a look at the accuracy of the 
 discussion. I expanded the "Workaround" section and moved it 
 near the top.


 I would also like to register that while I respect your 
 argument regarding scalability, I have personally found that a 
 greater number of smaller files is easier to manage than a 
 smaller number of larger files. Including for the 580,000+ line 
 project I work on for a living.


(Granted, that old ugly codebase has a lot of problems of its 
own, and smaller files does not always mean small on the same 
scale that mach.d's files are small, but I can tell you from 
experience that once a single ~4,000 line module was broken down 
into many ~200 line modules, that code became a great deal easier 
to understand and to reason about and to maintain.)
Dec 18 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/16 7:44 PM, pineapple wrote:

 On Sunday, 18 December 2016 at 23:18:27 UTC, Andrei Alexandrescu wrote:

 Great, thanks. Please take a look at the accuracy of the discussion. I
 expanded the "Workaround" section and moved it near the top.


 I would also like to register that while I respect your argument
 regarding scalability, I have personally found that a greater number of
 smaller files is easier to manage than a smaller number of larger files.
 Including for the 580,000+ line project I work on for a living.


Is it publicly available? -- Andrei
Dec 18 2016
parent pineapple <meapineapple gmail.com> writes:
On Monday, 19 December 2016 at 00:54:13 UTC, Andrei Alexandrescu 
wrote:

 On 12/18/16 7:44 PM, pineapple wrote:

 On Sunday, 18 December 2016 at 23:18:27 UTC, Andrei 
 Alexandrescu wrote:

 Great, thanks. Please take a look at the accuracy of the 
 discussion. I
 expanded the "Workaround" section and moved it near the top.


 I would also like to register that while I respect your 
 argument
 regarding scalability, I have personally found that a greater 
 number of
 smaller files is easier to manage than a smaller number of 
 larger files.
 Including for the 580,000+ line project I work on for a living.


 Is it publicly available? -- Andrei


Sadly not.
Dec 18 2016
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 18 December 2016 at 23:18:27 UTC, Andrei Alexandrescu 
wrote:

 Great, thanks. Please take a look at the accuracy of the 
 discussion. I expanded the "Workaround" section and moved it 
 near the top.

 https://github.com/dlang/DIPs/pull/51

 https://github.com/dlang/DIPs/blob/dd46252e820dce66df746540d7ab94e0b00a6505/DIPs/DIP1005.md


 Andrei


What's wrong with the parentheseless version ? The DIP says it 
looks "out of place" but that doesn't strike me as a very good 
argument. IMO the version without ";" is the way to go, as it 
doesn't require to add a new syntax for imports.

Identical things looking identical is valuable.
Dec 18 2016
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 19.12.2016 06:31, deadalnix wrote:

 On Sunday, 18 December 2016 at 23:18:27 UTC, Andrei Alexandrescu wrote:

 Great, thanks. Please take a look at the accuracy of the discussion. I
 expanded the "Workaround" section and moved it near the top.

 https://github.com/dlang/DIPs/pull/51

 https://github.com/dlang/DIPs/blob/dd46252e820dce66df746540d7ab94e0b00a6505/DIPs/DIP1005.md



 Andrei


 What's wrong with the parentheseless version ? The DIP says it looks
 "out of place" but that doesn't strike me as a very good argument. IMO
 the version without ";" is the way to go, as it doesn't require to add a
 new syntax for imports.

 Identical things looking identical is valuable.


+1.
Dec 20 2016
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2016-12-18 01:34, Andrei Alexandrescu wrote:


 Yeah, std/datetime.d is a monster, from what I can tell owing to a rote
 and redundant way of handling unittesting. I didn't look at its
 dependencies, but I doubt they are special. I was quite vocal about
 breaking it up, but I got mellower with time since (a) someone measured
 its size without unittests and it was something like one order of
 magnitude smaller, and (b) there was really no more trouble using or
 maintaining it than with anything else in Phobos.


Most other languages don't have inline unit tests, which saves a lot of 
lines of code.

Not sure if this is the case. But if we have unit tests that are more on 
the functional/integration side perhaps those should be moved to a 
separate file structure.


 I should also add that each large project has a couple of outliers like
 that. I even recall a switch of a couple thousand lines once :o).


Just because another project is worse doesn't mean we're in a good position.

Rubocop, the major linter in the Ruby world, will complain if a class is 
more than 100 lines of code. I think that is on the extreme side but I 
think any module with more than 2000 lines of code is too big.

-- 
/Jacob Carlborg
Dec 18 2016
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 18 December 2016 at 11:47:06 UTC, Jacob Carlborg wrote:

 I think any module with more than 2000 lines of code is too big.


newCTFE has more than 2000 lines of code and is still growing,
There is no way it could be split up.
Dec 18 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/16 6:47 AM, Jacob Carlborg wrote:

 On 2016-12-18 01:34, Andrei Alexandrescu wrote:


 Yeah, std/datetime.d is a monster, from what I can tell owing to a rote
 and redundant way of handling unittesting. I didn't look at its
 dependencies, but I doubt they are special. I was quite vocal about
 breaking it up, but I got mellower with time since (a) someone measured
 its size without unittests and it was something like one order of
 magnitude smaller, and (b) there was really no more trouble using or
 maintaining it than with anything else in Phobos.


 Most other languages don't have inline unit tests, which saves a lot of
 lines of code.


How does the lack of the feature save lines? Doesn't it just move them 
elsewhere?


 Not sure if this is the case. But if we have unit tests that are more on
 the functional/integration side perhaps those should be moved to a
 separate file structure.


 I should also add that each large project has a couple of outliers like
 that. I even recall a switch of a couple thousand lines once :o).


 Just because another project is worse doesn't mean we're in a good
 position.


The point was just that it's par for the course.


 Rubocop, the major linter in the Ruby world, will complain if a class is
 more than 100 lines of code.


Does that include full documentation and unittests?


 I think that is on the extreme side but I
 think any module with more than 2000 lines of code is too big.


Phobos' average file size 2055 is lines and median file size is 903 
lines. Do you find these appropriate for your preferences, considering 
this size includes documentation and unittests? Would you find it 
helpful to use a tool that collapses these?


Andrei
Dec 18 2016
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 12/18/2016 5:38 AM, Andrei Alexandrescu wrote:

 On 12/18/16 6:47 AM, Jacob Carlborg wrote:

 Rubocop, the major linter in the Ruby world, will complain if a class is
 more than 100 lines of code.


 Does that include full documentation and unittests?


My complaint is more along the lines of aggregates that seem to have scores of 
member functions.

It brings to mind Scott Meyers' article about kitchen sink aggregates:

http://www.drdobbs.com/cpp/how-non-member-functions-improve-encapsu/184401197
Dec 18 2016
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2016-12-18 14:38, Andrei Alexandrescu wrote:


 How does the lack of the feature save lines? Doesn't it just move them
 elsewhere?


Yes, I was referring to the file containing the implementation.


 Does that include full documentation and unittests?


It does not count the documentation in those lines. It would count the 
unit tests because there's no special syntax for that, just regular Ruby 
code. But one would most likely not have the implementation and the unit 
tests in the same file.


 Phobos' average file size 2055 is lines and median file size is 903
 lines. Do you find these appropriate for your preferences, considering
 this size includes documentation and unittests?


I would prefer smaller, but I would also prefer to have the unit tests 
separately.


 Would you find it helpful to use a tool that collapses these?


It's more helpful, yes. I do have that but since signatures in D can be 
arbitrary complex I'm doubtful that there's a tool/editor can fold all 
signatures that are valid in D. I know that TextMate that I use cannot. 
Take this for example:

typeof({
     class Foo
     {
         void bar()
         {
         }
     }
}) foo ();

I know that that's highly unlikely to happen in the real world but it 
shows how problematic it is to fold D code.

-- 
/Jacob Carlborg
Dec 19 2016
prev sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Thursday, 15 December 2016 at 22:56:42 UTC, Dmitry Olshansky 
wrote:

 On 12/13/16 11:33 PM, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


 On first it seems like an awesome idea. That solves ... but 
 wait what? Thinking more  about the problem at hand - I fail to 
 see what this DIP accomplishes that can't be done better today.

 1. The benefit of placing import to each function is based on 
 the untold assumption that we have:
 a) huge modules with many functions
 b) that constraints of these functions require different 
 (large) modules

 The reality is far from this picture - if anything 99% of 
 template constraints are dependent on std.range.primitives and 
 std.traits with a bit of std.meta from time to time. So we'd 
 have a boilerplate of

 auto foo(R)(R range)
 (import std.range.primitives)
 if(isInputRange!R){ ... }

 everywhere for no noticeable benefit - touch one of functions 
 and you get full set of imports of these _small_ modules.

 2. By itself the mechanism for delaying import even for 
 constraint until the function is touched is moot as long as the 
 module in question is huge and is not split in pieces. In other 
 words:

 auto foo(R)(R range)
 (import std.range)
 if(isInputRange!R){ ... }

 Pulls in full std.range the moment foo is touched, compared to

 import std.range.primitives;
 ...
 auto foo(R)(R range)
 if(isInputRange!R){ ... }

 which is because it actually isolates the whole mess of 
 complete std.range from the user of a template.

 All in all my practical response is split the modules at least 
 in 2 parts: constraints + full functionality, then import the 
 one with constraints at the top level. Works today and solves 
 the practical issues unlike the proposal.

 ---
 Dmitry Olshansky


I largely agree with Dmitry. Ilya refactored several Phobos 
modules to use scoped, selective imports much more, and I pitched 
in for some remaining imports in the largest modules, so that 
only these module-level imports remain, ie those necessary for 
symbols imported in template constraints:

std.datetime - https://github.com/dlang/phobos/pull/4373/files
std.uni - https://github.com/dlang/phobos/pull/4365/files
std.string and std.traits - 
https://github.com/dlang/phobos/pull/4370/files

When I first saw this DIP, like Dmitry I was happy that we could 
get rid of those too, but the more I see these horrible syntax 
suggestions for what is really a minor convenience, I changed my 
mind.  std.datetime, the 35k line (17 kloc according to Dscanner) 
beast of phobos, only needs 20 or so symbols at module-scope. 
std.uni- 10k lines, 4.2 kloc- only needs 17 symbols, all from the 
three modules Dmitry mentioned.  I don't think his workaround of 
splitting up modules is even needed for such a low amount of 
module-level imports.

Maybe there are other issues having to do with symbol resolution 
and dependency encapsulation that are addressed by this DIP, ie 
the technical performance of the compiler rather than refactoring 
or code clarity, that I don't fully grasp, but from the first two 
points of the claimed benefits of DCDs, ie ease of reasoning 
about dependencies and refactoring code, I don't think this 
feature will come anywhere close to carrying its own weight.

As for the third benefit having to do with scalable template 
libraries, I'm not sure I completely understand all the details 
there, but I wonder if those problems aren't an artifact of the 
way dmd works now rather than something that can't be fixed 
without this DIP.
Dec 18 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/16 1:03 PM, Joakim wrote:

 I largely agree with Dmitry. Ilya refactored several Phobos modules to
 use scoped, selective imports much more, and I pitched in for some
 remaining imports in the largest modules, so that only these
 module-level imports remain, ie those necessary for symbols imported in
 template constraints:

 std.datetime - https://github.com/dlang/phobos/pull/4373/files
 std.uni - https://github.com/dlang/phobos/pull/4365/files
 std.string and std.traits - https://github.com/dlang/phobos/pull/4370/files


Yah, there's been a lot of good work (Jack Stouffer did a lot as well 
IIRC) on pushing imports inside. The following searches should be relevant:

git grep '^\(private \)\?import' | wc -l

yields about 426 top-level import declarations. The number of indented 
imports is 4605:

git grep '     *import\W' | wc -l

So we're looking at 10% of imports being problematic. Sadly, they turn 
out to make things quite difficult for Phobos. Last time I looked at the 
module dependency graph it wasn't a lot better than it used to before 
scoped imports. (I don't have it handy... could anyone please produce it?)


 When I first saw this DIP, like Dmitry I was happy that we could get rid
 of those too, but the more I see these horrible syntax suggestions for
 what is really a minor convenience, I changed my mind.  std.datetime,
 the 35k line (17 kloc according to Dscanner) beast of phobos, only needs
 20 or so symbols at module-scope. std.uni- 10k lines, 4.2 kloc- only
 needs 17 symbols, all from the three modules Dmitry mentioned.  I don't
 think his workaround of splitting up modules is even needed for such a
 low amount of module-level imports.


This paragraph is a good example of a couple of counterarguments that I 
think point directly to flaws in the DIP:

(1) The DIP uses Phobos as an example, so it is applicable mostly to 
Phobos. In fact, Phobos is among the systems that would benefit least 
from the DIP: it has only druntime as dependency, and is distributed in 
its entirety. Many projects out there list multiple dependencies and may 
have various building and distribution policies. The converse is to 
believe that working around a problem in Phobos would render the DIP 
less useful in general.

(2) "I don't like the syntax, hence I don't like the feature." I see 
this as a good opportunity for tasteful design, not a downside of the 
feature.


 Maybe there are other issues having to do with symbol resolution and
 dependency encapsulation that are addressed by this DIP, ie the
 technical performance of the compiler rather than refactoring or code
 clarity, that I don't fully grasp, but from the first two points of the
 claimed benefits of DCDs, ie ease of reasoning about dependencies and
 refactoring code, I don't think this feature will come anywhere close to
 carrying its own weight.


Does the refactoring in https://github.com/dlang/phobos/pull/4962 make 
dependencies clearer? Are you e.g. clear on what you need in order to 
use Appender? Would you want to take this (or another) experiment to 
another module and see how it improves its dependency structure?


 As for the third benefit having to do with scalable template libraries,
 I'm not sure I completely understand all the details there, but I wonder
 if those problems aren't an artifact of the way dmd works now rather
 than something that can't be fixed without this DIP.


The DIP now dedicates an entire section to the pluses and minuses of 
lazy imports, and concludes that lazy imports would address scalability 
if carefully used.


Andrei
Dec 18 2016
parent reply Joakim <dlang joakim.fea.st> writes:
On Sunday, 18 December 2016 at 18:42:36 UTC, Andrei Alexandrescu 
wrote:

 On 12/18/16 1:03 PM, Joakim wrote:

 I largely agree with Dmitry. Ilya refactored several Phobos 
 modules to
 use scoped, selective imports much more, and I pitched in for 
 some
 remaining imports in the largest modules, so that only these
 module-level imports remain, ie those necessary for symbols 
 imported in
 template constraints:

 std.datetime - https://github.com/dlang/phobos/pull/4373/files
 std.uni - https://github.com/dlang/phobos/pull/4365/files
 std.string and std.traits - 
 https://github.com/dlang/phobos/pull/4370/files


 Yah, there's been a lot of good work (Jack Stouffer did a lot 
 as well IIRC) on pushing imports inside. The following searches 
 should be relevant:

 git grep '^\(private \)\?import' | wc -l

 yields about 426 top-level import declarations. The number of 
 indented imports is 4605:

 git grep '     *import\W' | wc -l

 So we're looking at 10% of imports being problematic. Sadly, 
 they turn out to make things quite difficult for Phobos. Last 
 time I looked at the module dependency graph it wasn't a lot 
 better than it used to before scoped imports. (I don't have it 
 handy... could anyone please produce it?)


Why do you care _so_ much about the module dependency graph?  To 
make this question concrete, let's look at an example, the 
std.array module you keep mentioning.  This is what it looked 
like before Ilya scoped as many imports as he could:

https://github.com/dlang/phobos/commit/3fcf723aa498b96de165361b5abb9d3450fdc069#diff-54cf8402b22024ae667d4048a5126f0e

That was a mess, similar to opaque C/C++ code, 13 modules 
imported at module-scope were whittled down to 4.  You just made 
those more specific in a commit related to this DIP, by listing 
the actual symbols selectively imported from those four modules:

https://github.com/dlang/phobos/commit/e064d5664f92c4b2f0866c08f6d0290ba66825ed#diff-54cf8402b22024ae667d4048a5126f0e

If I'm looking at the template constraints for any particular 
function and see a couple symbols I don't recognize, I don't 
think it's a big deal to find the symbols in that list at the top.

In other words, D already allows you to scope most imports.  I 
don't consider the dozen or two remaining symbols from templaint 
constraints and function arguments to provide much overhead.  
Rather, I consider the weight of this additional syntax, ie the 
cognitive overhead from having to remember and parse more syntax 
in my head, to be worse than the remaining dependency reasoning 
problem you're trying to solve: the cost outweights the benefit.  
Perhaps that's subjective and others may disagree.

Now, there's also the question of purely technical benefits, like 
compilation speed or executable bloat.  I looked at the latter a 
little last summer, after Ilya had cleaned up a lot of the 
standard library:

http://forum.dlang.org/thread/gmjqfjoemwtvgqrtdsdr forum.dlang.org

I found that commenting out a single scoped, selective import of 
"std.string: format" in std.utf led to a 5% decrease in 
executable size for "hello world."  This is a problem with how 
dmd compiles or appends these module dependencies and would 
presumably still be there after this DIP, as you would not remove 
the dependency.

I think scoped, selective imports have been great at hacking away 
at the module dependency graph, as you lay out.  It is not clear 
what technical costs you see from the remaining few dependencies 
and if this DIP is the best way to remove them.  I think you 
should explain why you want to untangle the remaining dependency 
graph, and consider if this DIP is really doing that much.


 When I first saw this DIP, like Dmitry I was happy that we 
 could get rid
 of those too, but the more I see these horrible syntax 
 suggestions for
 what is really a minor convenience, I changed my mind.  
 std.datetime,
 the 35k line (17 kloc according to Dscanner) beast of phobos, 
 only needs
 20 or so symbols at module-scope. std.uni- 10k lines, 4.2 
 kloc- only
 needs 17 symbols, all from the three modules Dmitry mentioned.
  I don't
 think his workaround of splitting up modules is even needed 
 for such a
 low amount of module-level imports.


 This paragraph is a good example of a couple of 
 counterarguments that I think point directly to flaws in the 
 DIP:

 (1) The DIP uses Phobos as an example, so it is applicable 
 mostly to Phobos. In fact, Phobos is among the systems that 
 would benefit least from the DIP: it has only druntime as 
 dependency, and is distributed in its entirety. Many projects 
 out there list multiple dependencies and may have various 
 building and distribution policies.


It is not clear how those alternate dependency, building and 
distribution policies change the picture.  Perhaps you should 
cite one of those as an example.


 The converse is to believe that working around a problem in 
 Phobos would render the DIP less useful in general.


The argument is not that Phobos has "worked around" the problem, 
but that once you scope everything you can other than template 
constraints (which you have said should be standard operating 
procedure in the DIP), the problem is minimal.


 (2) "I don't like the syntax, hence I don't like the feature." 
 I see this as a good opportunity for tasteful design, not a 
 downside of the feature.


The syntax is bad, but even if were beautiful, I don't consider 
the addition itself to be worth it.  Again, could be subjective.


 Maybe there are other issues having to do with symbol 
 resolution and
 dependency encapsulation that are addressed by this DIP, ie the
 technical performance of the compiler rather than refactoring 
 or code
 clarity, that I don't fully grasp, but from the first two 
 points of the
 claimed benefits of DCDs, ie ease of reasoning about 
 dependencies and
 refactoring code, I don't think this feature will come 
 anywhere close to
 carrying its own weight.


 Does the refactoring in 
 https://github.com/dlang/phobos/pull/4962 make dependencies 
 clearer?


I'm fine with your selective import refactoring, which I linked 
above:

https://github.com/dlang/phobos/pull/4963


 Are you e.g. clear on what you need in order to use Appender?


It is easy to see that it requires isDynamicArray and that is 
found in std.traits.  It may internally rely on other symbols 
that are currently imported from module scope, ie symbols from 
template constraints inside the struct Appender that happen to 
also be used outside that struct and so are imported at the top 
of the module.  If you really wanted to make that clear, you 
could scope those imports inside the struct too.


 Would you want to take this (or another) experiment to another 
 module and see how it improves its dependency structure?


Ilya and I have already done this, in the PRs I linked above.  I 
was mildly disappointed back then that I could not get rid of the 
remaining module-level imports, because of template constraints.  
I was predisposed to favor doing something about it, but I don't 
think this DIP is worth it.


 As for the third benefit having to do with scalable template 
 libraries,
 I'm not sure I completely understand all the details there, 
 but I wonder
 if those problems aren't an artifact of the way dmd works now 
 rather
 than something that can't be fixed without this DIP.


 The DIP now dedicates an entire section to the pluses and 
 minuses of lazy imports, and concludes that lazy imports would 
 address scalability if carefully used.


Yes, there might be other ways to lower the technical costs of 
the dependency graph too, such as the size issue I mentioned 
above.
Dec 18 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/19/2016 01:47 AM, Joakim wrote:

 Why do you care _so_ much about the module dependency graph?  To make
 this question concrete, let's look at an example, the std.array module
 you keep mentioning.
 This is what it looked like before Ilya scoped as
 many imports as he could:

 https://github.com/dlang/phobos/commit/3fcf723aa498b96de165361b5abb9d3450fdc069#diff-54cf8402b22024ae667d4048a5126f0e

 That was a mess, similar to opaque C/C++ code, 13 modules imported at
 module-scope were whittled down to 4.  You just made those more specific
 in a commit related to this DIP, by listing the actual symbols
 selectively imported from those four modules:

 https://github.com/dlang/phobos/commit/e064d5664f92c4b2f0866c08f6d0290ba66825ed#diff-54cf8402b22024ae667d4048a5126f0e


 If I'm looking at the template constraints for any particular function
 and see a couple symbols I don't recognize, I don't think it's a big
 deal to find the symbols in that list at the top.


It actually is. Some symbols in e.g. std.range are used in constraints, 
some others used in definitions; some in a UFCS manner. Some code runs 
during compilation, with errors gagged (as in __traits(compiles)) or not 
gagged. I don't know of an easy manual method to figure out who's who 
(which makes Vladimir's idea of tooling it with brute force awesome and 
scary at the same time).

At any way, I don't see how I can use this tidbit to improve the DIP. 
Anything I could add to it to make it more compelling?


 In other words, D already allows you to scope most imports.  I don't
 consider the dozen or two remaining symbols from templaint constraints
 and function arguments to provide much overhead.  Rather, I consider the
 weight of this additional syntax, ie the cognitive overhead from having
 to remember and parse more syntax in my head, to be worse than the
 remaining dependency reasoning problem you're trying to solve: the cost
 outweights the benefit.  Perhaps that's subjective and others may disagree.


It is subjective and difficult to convert into action. Online review 
among folks who know next to nothing about each other does have its 
challenges. What happens here is I post a proposal for a problem that I 
believe is important for large D projects. And I get back "eh, that's 
not as important to me." At a traditional work place we'd know a lot 
about one another's strengths, specialization areas, and design 
sensibilities. Here, all I can do is look at your past PRs (that's why I 
emailed you asking for your github handle; I figure it's  joakim-noah). 
This makes it difficult to act on "I don't buy it" feedback.


 Now, there's also the question of purely technical benefits, like
 compilation speed or executable bloat.  I looked at the latter a little
 last summer, after Ilya had cleaned up a lot of the standard library:

 http://forum.dlang.org/thread/gmjqfjoemwtvgqrtdsdr forum.dlang.org

 I found that commenting out a single scoped, selective import of
 "std.string: format" in std.utf led to a 5% decrease in executable size
 for "hello world."  This is a problem with how dmd compiles or appends
 these module dependencies and would presumably still be there after this
 DIP, as you would not remove the dependency.


That might be a related but distinct issue. Can you reproduce that 
experiment?


 I think scoped, selective imports have been great at hacking away at the
 module dependency graph, as you lay out.  It is not clear what technical
 costs you see from the remaining few dependencies and if this DIP is the
 best way to remove them.  I think you should explain why you want to
 untangle the remaining dependency graph, and consider if this DIP is
 really doing that much.


I've made a few more passes through the DIP, but throughout I assume a 
couple of things without stressing them too much - dependencies are 
important, and associating dependencies with declarations makes the 
dependency graph as fine and as precise it could get. I don't see a 
reasonable way to make it better or clearer.

Should I add an introductory section on why dependencies are important?


Andrei
Dec 19 2016
next sibling parent Timothee Cour via Digitalmars-d <digitalmars-d puremagic.com> writes:
Andrei: you can use blockquotes to make long lines wrap when quoting text:

https://github.com/adam-p/markdown-here/wiki/Markdown-Here-Cheatsheet#blockquotes

that way, we don't have to scroll everytime there's a quote, eg:

replace:
<tab>Such scaffolding is of course
with:

 Such scaffolding is of course





On Mon, Dec 19, 2016 at 8:04 PM, Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 12/19/2016 01:47 AM, Joakim wrote:


 Why do you care _so_ much about the module dependency graph?  To make
 this question concrete, let's look at an example, the std.array module
 you keep mentioning.
 This is what it looked like before Ilya scoped as
 many imports as he could:

 https://github.com/dlang/phobos/commit/3fcf723aa498b96de1653
 61b5abb9d3450fdc069#diff-54cf8402b22024ae667d4048a5126f0e

 That was a mess, similar to opaque C/C++ code, 13 modules imported at
 module-scope were whittled down to 4.  You just made those more specific
 in a commit related to this DIP, by listing the actual symbols
 selectively imported from those four modules:

 https://github.com/dlang/phobos/commit/e064d5664f92c4b2f0866
 c08f6d0290ba66825ed#diff-54cf8402b22024ae667d4048a5126f0e


 If I'm looking at the template constraints for any particular function
 and see a couple symbols I don't recognize, I don't think it's a big
 deal to find the symbols in that list at the top.


 It actually is. Some symbols in e.g. std.range are used in constraints,
 some others used in definitions; some in a UFCS manner. Some code runs
 during compilation, with errors gagged (as in __traits(compiles)) or not
 gagged. I don't know of an easy manual method to figure out who's who
 (which makes Vladimir's idea of tooling it with brute force awesome and
 scary at the same time).

 At any way, I don't see how I can use this tidbit to improve the DIP.
 Anything I could add to it to make it more compelling?

 In other words, D already allows you to scope most imports.  I don't

 consider the dozen or two remaining symbols from templaint constraints
 and function arguments to provide much overhead.  Rather, I consider the
 weight of this additional syntax, ie the cognitive overhead from having
 to remember and parse more syntax in my head, to be worse than the
 remaining dependency reasoning problem you're trying to solve: the cost
 outweights the benefit.  Perhaps that's subjective and others may
 disagree.


 It is subjective and difficult to convert into action. Online review among
 folks who know next to nothing about each other does have its challenges.
 What happens here is I post a proposal for a problem that I believe is
 important for large D projects. And I get back "eh, that's not as important
 to me." At a traditional work place we'd know a lot about one another's
 strengths, specialization areas, and design sensibilities. Here, all I can
 do is look at your past PRs (that's why I emailed you asking for your
 github handle; I figure it's  joakim-noah). This makes it difficult to act
 on "I don't buy it" feedback.

 Now, there's also the question of purely technical benefits, like

 compilation speed or executable bloat.  I looked at the latter a little
 last summer, after Ilya had cleaned up a lot of the standard library:

 http://forum.dlang.org/thread/gmjqfjoemwtvgqrtdsdr forum.dlang.org

 I found that commenting out a single scoped, selective import of
 "std.string: format" in std.utf led to a 5% decrease in executable size
 for "hello world."  This is a problem with how dmd compiles or appends
 these module dependencies and would presumably still be there after this
 DIP, as you would not remove the dependency.


 That might be a related but distinct issue. Can you reproduce that
 experiment?

 I think scoped, selective imports have been great at hacking away at the

 module dependency graph, as you lay out.  It is not clear what technical
 costs you see from the remaining few dependencies and if this DIP is the
 best way to remove them.  I think you should explain why you want to
 untangle the remaining dependency graph, and consider if this DIP is
 really doing that much.


 I've made a few more passes through the DIP, but throughout I assume a
 couple of things without stressing them too much - dependencies are
 important, and associating dependencies with declarations makes the
 dependency graph as fine and as precise it could get. I don't see a
 reasonable way to make it better or clearer.

 Should I add an introductory section on why dependencies are important?


 Andrei
Dec 19 2016
prev sibling next sibling parent Timothee Cour via Digitalmars-d <digitalmars-d puremagic.com> writes:
what about using `lazy` instead of `with`:

`with(import foo)`
=>
`lazy(import foo)`

advantages:
* avoids confusion regarding usual scoping rules of `with` ;
* conveys that the import is indeed lazy

Furthermore (regardless of which keyword is used), what about allowing `:`
```
// case 1
lazy(import foo)
void fun(){}

// case 2
lazy(import foo) {
  void fun(){}
}

// case 3 : this is new
lazy(import foo):
void fun1(){}
void fun2(){}
```

advantages:

* same behavior as other constructs which don't introduce a scope:
```
// case 1, 2 3 are allowed:
version(A):
static if(true):
private:
void fun(){}
```

* avoids nesting when case 3 is used (compared to when using `{}`)

* I would argue that grouping lazy imports is actually a common case;
without case 3, the indentation will increase.






On Mon, Dec 19, 2016 at 8:56 PM, Timothee Cour <thelastmammoth gmail.com>
wrote:


 Andrei: you can use blockquotes to make long lines wrap when quoting text:

 https://github.com/adam-p/markdown-here/wiki/Markdown-
 Here-Cheatsheet#blockquotes

 that way, we don't have to scroll everytime there's a quote, eg:

 replace:
 <tab>Such scaffolding is of course
 with:

 Such scaffolding is of course





 On Mon, Dec 19, 2016 at 8:04 PM, Andrei Alexandrescu via Digitalmars-d <
 digitalmars-d puremagic.com> wrote:


 On 12/19/2016 01:47 AM, Joakim wrote:


 Why do you care _so_ much about the module dependency graph?  To make
 this question concrete, let's look at an example, the std.array module
 you keep mentioning.
 This is what it looked like before Ilya scoped as
 many imports as he could:

 https://github.com/dlang/phobos/commit/3fcf723aa498b96de1653
 61b5abb9d3450fdc069#diff-54cf8402b22024ae667d4048a5126f0e

 That was a mess, similar to opaque C/C++ code, 13 modules imported at
 module-scope were whittled down to 4.  You just made those more specific
 in a commit related to this DIP, by listing the actual symbols
 selectively imported from those four modules:

 https://github.com/dlang/phobos/commit/e064d5664f92c4b2f0866
 c08f6d0290ba66825ed#diff-54cf8402b22024ae667d4048a5126f0e


 If I'm looking at the template constraints for any particular function
 and see a couple symbols I don't recognize, I don't think it's a big
 deal to find the symbols in that list at the top.


 It actually is. Some symbols in e.g. std.range are used in constraints,
 some others used in definitions; some in a UFCS manner. Some code runs
 during compilation, with errors gagged (as in __traits(compiles)) or not
 gagged. I don't know of an easy manual method to figure out who's who
 (which makes Vladimir's idea of tooling it with brute force awesome and
 scary at the same time).

 At any way, I don't see how I can use this tidbit to improve the DIP.
 Anything I could add to it to make it more compelling?

 In other words, D already allows you to scope most imports.  I don't

 consider the dozen or two remaining symbols from templaint constraints
 and function arguments to provide much overhead.  Rather, I consider the
 weight of this additional syntax, ie the cognitive overhead from having
 to remember and parse more syntax in my head, to be worse than the
 remaining dependency reasoning problem you're trying to solve: the cost
 outweights the benefit.  Perhaps that's subjective and others may
 disagree.


 It is subjective and difficult to convert into action. Online review
 among folks who know next to nothing about each other does have its
 challenges. What happens here is I post a proposal for a problem that I
 believe is important for large D projects. And I get back "eh, that's not
 as important to me." At a traditional work place we'd know a lot about one
 another's strengths, specialization areas, and design sensibilities. Here,
 all I can do is look at your past PRs (that's why I emailed you asking for
 your github handle; I figure it's  joakim-noah). This makes it difficult to
 act on "I don't buy it" feedback.

 Now, there's also the question of purely technical benefits, like

 compilation speed or executable bloat.  I looked at the latter a little
 last summer, after Ilya had cleaned up a lot of the standard library:

 http://forum.dlang.org/thread/gmjqfjoemwtvgqrtdsdr forum.dlang.org

 I found that commenting out a single scoped, selective import of
 "std.string: format" in std.utf led to a 5% decrease in executable size
 for "hello world."  This is a problem with how dmd compiles or appends
 these module dependencies and would presumably still be there after this
 DIP, as you would not remove the dependency.


 That might be a related but distinct issue. Can you reproduce that
 experiment?

 I think scoped, selective imports have been great at hacking away at the

 module dependency graph, as you lay out.  It is not clear what technical
 costs you see from the remaining few dependencies and if this DIP is the
 best way to remove them.  I think you should explain why you want to
 untangle the remaining dependency graph, and consider if this DIP is
 really doing that much.


 I've made a few more passes through the DIP, but throughout I assume a
 couple of things without stressing them too much - dependencies are
 important, and associating dependencies with declarations makes the
 dependency graph as fine and as precise it could get. I don't see a
 reasonable way to make it better or clearer.

 Should I add an introductory section on why dependencies are important?


 Andrei
Dec 19 2016
prev sibling next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Tuesday, 20 December 2016 at 04:04:14 UTC, Andrei Alexandrescu 
wrote:

 On 12/19/2016 01:47 AM, Joakim wrote:

 Why do you care _so_ much about the module dependency graph?  
 To make
 this question concrete, let's look at an example, the 
 std.array module
 you keep mentioning.
 This is what it looked like before Ilya scoped as
 many imports as he could:

 https://github.com/dlang/phobos/commit/3fcf723aa498b96de165361b5abb9d3450fdc069#diff-54cf8402b22024ae667d4048a5126f0e

 That was a mess, similar to opaque C/C++ code, 13 modules 
 imported at
 module-scope were whittled down to 4.  You just made those 
 more specific
 in a commit related to this DIP, by listing the actual symbols
 selectively imported from those four modules:

 https://github.com/dlang/phobos/commit/e064d5664f92c4b2f0866c08f6d0290ba66825ed#diff-54cf8402b22024ae667d4048a5126f0e


 If I'm looking at the template constraints for any particular 
 function
 and see a couple symbols I don't recognize, I don't think it's 
 a big
 deal to find the symbols in that list at the top.


 It actually is. Some symbols in e.g. std.range are used in 
 constraints, some others used in definitions; some in a UFCS 
 manner. Some code runs during compilation, with errors gagged 
 (as in __traits(compiles)) or not gagged. I don't know of an 
 easy manual method to figure out who's who (which makes 
 Vladimir's idea of tooling it with brute force awesome and 
 scary at the same time).


I'm not sure why you care who's who.  I noted that if I'm looking 
at the constraints for a function, it's easy to check the 
selective imports at the top of the file for any symbols.  That's 
the common scenario.  I don't know why anyone would be checking 
the selective imports at the top to see where each symbol is 
actually used, which is the uncommon scenario you now present.


 At any way, I don't see how I can use this tidbit to improve 
 the DIP. Anything I could add to it to make it more compelling?


You could answer my question above. We have already scoped 90% of 
the dependencies, why is it so important to remove the remaining 
10% that we need to add new syntax?


 In other words, D already allows you to scope most imports.  I 
 don't
 consider the dozen or two remaining symbols from templaint 
 constraints
 and function arguments to provide much overhead.  Rather, I 
 consider the
 weight of this additional syntax, ie the cognitive overhead 
 from having
 to remember and parse more syntax in my head, to be worse than 
 the
 remaining dependency reasoning problem you're trying to solve: 
 the cost
 outweights the benefit.  Perhaps that's subjective and others 
 may disagree.


 It is subjective and difficult to convert into action. Online 
 review among folks who know next to nothing about each other 
 does have its challenges. What happens here is I post a 
 proposal for a problem that I believe is important for large D 
 projects. And I get back "eh, that's not as important to me." 
 At a traditional work place we'd know a lot about one another's 
 strengths, specialization areas, and design sensibilities. 
 Here, all I can do is look at your past PRs (that's why I 
 emailed you asking for your github handle; I figure it's 
  joakim-noah). This makes it difficult to act on "I don't buy 
 it" feedback.


I didn't just say "eh:" I gave evidence for why I think the 
problem is minimal and asked why it's so important to scope those 
last 3-4 imported modules too, which you didn't answer.

As for looking at my PRs, there were some links above, which show 
that is my handle, but I don't think you'll get much from that, 
as I haven't designed anything in github.com/dlang, only cleaning 
up and porting.


 Now, there's also the question of purely technical benefits, 
 like
 compilation speed or executable bloat.  I looked at the latter 
 a little
 last summer, after Ilya had cleaned up a lot of the standard 
 library:

 http://forum.dlang.org/thread/gmjqfjoemwtvgqrtdsdr forum.dlang.org

 I found that commenting out a single scoped, selective import 
 of
 "std.string: format" in std.utf led to a 5% decrease in 
 executable size
 for "hello world."  This is a problem with how dmd compiles or 
 appends
 these module dependencies and would presumably still be there 
 after this
 DIP, as you would not remove the dependency.


 That might be a related but distinct issue. Can you reproduce 
 that experiment?


I went back and looked at that particular import and it appears 
Walter subsequently removed it:

https://github.com/dlang/phobos/pull/3455

I will experiment a bit more with some sample code and see what I 
find.


 I think scoped, selective imports have been great at hacking 
 away at the
 module dependency graph, as you lay out.  It is not clear what 
 technical
 costs you see from the remaining few dependencies and if this 
 DIP is the
 best way to remove them.  I think you should explain why you 
 want to
 untangle the remaining dependency graph, and consider if this 
 DIP is
 really doing that much.


 I've made a few more passes through the DIP, but throughout I 
 assume a couple of things without stressing them too much - 
 dependencies are important, and associating dependencies with 
 declarations makes the dependency graph as fine and as precise 
 it could get. I don't see a reasonable way to make it better or 
 clearer.

 Should I add an introductory section on why dependencies are 
 important?


If it deals with technical costs of dependencies that you're 
aiming to alleviate with this DIP, yes.  I don't see why the 
module constructor reason you just gave can't be automated.  We 
appear to disagree on the value of localizing the remaining 
selective imports needed for constraints/declarations, from the 
standpoint of reasoning and refactoring.

Let me note that I'm not the only one in this thread saying that 
this isn't a problem worth adding syntax for and wondering why 
this DIP is worth doing.

If we're going to do it, I think Dominik's suggestion is the 
best: just change the semantics of scoped imports to apply to the 
constraints and function arguments too.  I don't think it's as 
surprising as you think, as the constraints are part of the 
function/struct.  If somebody buries the import deep in the 
function and far away from the constraint, that's just bad style 
on their part, and is already possible (and annoying) now.  We 
can automate the imports for declarations or leave that case 
alone.

Finally, while all this dependency-localizing support is good, we 
really need tools to help us do it.  I just happened to be 
thinking again about implementing such a tool a couple days 
before you posted this DIP.  It'll have to wait till I push out 
the latest Android beta in the next week or two.
Dec 20 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
I've asked Joakim about this via email just now, likely other folks also 
know the answer:

1. I found these PRs related to local imports:

https://github.com/dlang/phobos/pull/4361
https://github.com/dlang/phobos/pull/4365
https://github.com/dlang/phobos/pull/4370
https://github.com/dlang/phobos/pull/4373
https://github.com/dlang/phobos/pull/4379
https://github.com/dlang/phobos/pull/4392
https://github.com/dlang/phobos/pull/4467

Are there more? Is there a central point for them (such as a bugzilla 
issue)?

2. I see you've done a bunch of work in the area. Where, in your
estimate, are we on the spectrum of making imports local without a
major reorganization? Any low-hanging fruit to look at, or have Joakim,
Ilya, Jack and others made a pass through most/all modules already?


Thanks!
Dec 20 2016
next sibling parent Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Tuesday, 20 December 2016 at 20:51:54 UTC, Andrei Alexandrescu 
wrote:

 I've asked Joakim about this via email just now, likely other 
 folks also know the answer:

 1. I found these PRs related to local imports:

 https://github.com/dlang/phobos/pull/4361
 https://github.com/dlang/phobos/pull/4365
 https://github.com/dlang/phobos/pull/4370
 https://github.com/dlang/phobos/pull/4373
 https://github.com/dlang/phobos/pull/4379
 https://github.com/dlang/phobos/pull/4392
 https://github.com/dlang/phobos/pull/4467

 Are there more? Is there a central point for them (such as a 
 bugzilla issue)?

 2. I see you've done a bunch of work in the area. Where, in your
 estimate, are we on the spectrum of making imports local 
 without a
 major reorganization? Any low-hanging fruit to look at, or have 
 Joakim,
 Ilya, Jack and others made a pass through most/all modules 
 already?


 Thanks!


This is 95% of my PRs for local imports. Few of them moves codes 
between files.

https://github.com/dlang/phobos/pull/2658
https://github.com/dlang/phobos/pull/2659
https://github.com/dlang/phobos/pull/2661
https://github.com/dlang/phobos/pull/2665
https://github.com/dlang/phobos/pull/2666
https://github.com/dlang/phobos/pull/2667
https://github.com/dlang/phobos/pull/2669
https://github.com/dlang/phobos/pull/2670
https://github.com/dlang/phobos/pull/2671
https://github.com/dlang/phobos/pull/2672
https://github.com/dlang/phobos/pull/2673
https://github.com/dlang/phobos/pull/2675
https://github.com/dlang/phobos/pull/2678
https://github.com/dlang/phobos/pull/2679
https://github.com/dlang/phobos/pull/2680
https://github.com/dlang/phobos/pull/2681
https://github.com/dlang/phobos/pull/2686
https://github.com/dlang/phobos/pull/2691
https://github.com/dlang/phobos/pull/2694
https://github.com/dlang/phobos/pull/2695
https://github.com/dlang/phobos/pull/2696
https://github.com/dlang/phobos/pull/2705
https://github.com/dlang/phobos/pull/2706
https://github.com/dlang/phobos/pull/2707
https://github.com/dlang/phobos/pull/2708
https://github.com/dlang/phobos/pull/2709
https://github.com/dlang/phobos/pull/2710
https://github.com/dlang/phobos/pull/2711
https://github.com/dlang/phobos/pull/2712
https://github.com/dlang/phobos/pull/2713
https://github.com/dlang/phobos/pull/2714
https://github.com/dlang/phobos/pull/2715
https://github.com/dlang/phobos/pull/2716
https://github.com/dlang/phobos/pull/2717
https://github.com/dlang/phobos/pull/2718
https://github.com/dlang/phobos/pull/2719
https://github.com/dlang/phobos/pull/2720
https://github.com/dlang/phobos/pull/2721
https://github.com/dlang/phobos/pull/2722
https://github.com/dlang/phobos/pull/2725
https://github.com/dlang/phobos/pull/2726
https://github.com/dlang/phobos/pull/2727
https://github.com/dlang/phobos/pull/2728
https://github.com/dlang/phobos/pull/2729
https://github.com/dlang/phobos/pull/2754
https://github.com/dlang/phobos/pull/2755

Ilya
Dec 20 2016
prev sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Tuesday, 20 December 2016 at 20:51:54 UTC, Andrei Alexandrescu 
wrote:

 I've asked Joakim about this via email just now, likely other 
 folks also know the answer:

 1. I found these PRs related to local imports:

 https://github.com/dlang/phobos/pull/4361
 https://github.com/dlang/phobos/pull/4365
 https://github.com/dlang/phobos/pull/4370
 https://github.com/dlang/phobos/pull/4373
 https://github.com/dlang/phobos/pull/4379
 https://github.com/dlang/phobos/pull/4392
 https://github.com/dlang/phobos/pull/4467

 Are there more? Is there a central point for them (such as a 
 bugzilla issue)?


Ilya lists a lot more above, he did most of the work.  No central 
point that I know of.


 2. I see you've done a bunch of work in the area. Where, in your
 estimate, are we on the spectrum of making imports local 
 without a
 major reorganization? Any low-hanging fruit to look at, or have 
 Joakim,
 Ilya, Jack and others made a pass through most/all modules 
 already?


There is more to be done, but my guess would be 80-90% done for 
Phobos. Ilya scoped a lot, but usually left druntime imports 
alone.  Top-level module imports mostly don't use selective 
imports yet, because Martin wanted to hold off till he was sure 
the symbol leak bug was fixed.

On Tuesday, 20 December 2016 at 22:08:38 UTC, Andrei Alexandrescu 
wrote:

 On 12/20/2016 03:46 AM, Joakim wrote:

 I didn't just say "eh:" I gave evidence for why I think the 
 problem is
 minimal and asked why it's so important to scope those last 
 3-4 imported
 modules too, which you didn't answer.


 You have asked for a smoking gun, and one has been found.

 I have just uploaded a major update that carefully analyzes the 
 improvements brought about by switching to local imports in the 
 D Standard Library. Please refer to the section "Workaround: 
 Are Local Imports Good Enough?" and Appendix A:

 https://github.com/dlang/DIPs/pull/51/files

 https://github.com/dlang/DIPs/blob/91baecedcfe7cb75ac22e66478722ec797ebb901/DIPs/DIP1005.md


Thanks for this analysis of the remaining dependency graph, it is 
worth looking at.  Allow me to poke some holes in it.

To begin with, the amount of scoping that has been done is 
overstated, if you simply count scoped imports and compare it to 
module-level imports.  Each module-level import has to be 
replicated multiple times for each local scope, especially in 
unittest blocks.  A better number is more like 20-30%, as I 
pointed out 4 out of 13 modules remain at top-level in std.array. 
Using that metric, a 3-4X reduction in top-level imports has led 
to at least a 2.2x improvement in imported files, so the effort 
has been more meaningful than you conclude.

Second, as I noted above, most top-level imports have not been 
made selective yet, because of the symbol leak bug that was 
recently fixed by Martin.  You will see in my PRs that I only 
list those symbols as a comment, because I could not turn those 
into selective imports yet.  If the compiler is doing its job 
right, selective imports should greatly reduce the cost of 
importing a module, even if your metric would still show the 
module being imported.

Third, checking some of the output from the commands you ran in 
your script shows that up to half of the imported modules are 
from druntime.  I noted earlier that Ilya usually didn't bother 
scoping top-level druntime imports, because he perceived their 
cost to be low (I scoped them too in the handful I cleaned up, 
just for completeness).  As far as I know, nobody has bothered to 
spend any time scoping druntime, so it would be better if you 
filtered out druntime imports from your analysis.

Finally, while it's nice to know the extent of the dependency 
graph, what really matters is the _cost_ of each link of the 
graph, which is what I keep hammering on.  If the cost of links 
is small, it doesn't matter how entangled it is.  If minimizing 
the dependency graph through scoping alone, ie without 
implementing this DIP, removes most of the cost, that's all I 
care about.

I have noted one example above, where _a single DCD in phobos_, 
ie a scoped, selective import, had gigantic costs in terms of 
executable size, where entire modules were included because of 
it.  If that's the case more generally, then _no_ amount of 
dependency disentangling will matter, because the cost of single 
DCDs is still huge.  Perhaps that's just an isolated issue 
however, it needs to be investigated.

My point is that the dependency graph matters, but now that we're 
getting down to the last entanglements, we need to know the cost 
of those last links.  Your dependency analysis gives us some 
quantitative idea of the size of the remaining graph, but tells 
us nothing about the cost of those links. That's what I'm looking 
for.

I will spend some time now investigating those costs with sample 
code.  My request all along has been that you give us some idea 
of those costs, if you know the answer already.
Dec 20 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/20/2016 11:32 PM, Joakim wrote:

 On Tuesday, 20 December 2016 at 20:51:54 UTC, Andrei Alexandrescu wrote:
 Thanks for this analysis of the remaining dependency graph, it is worth
 looking at.  Allow me to poke some holes in it.

 To begin with, the amount of scoping that has been done is overstated,
 if you simply count scoped imports and compare it to module-level
 imports.  Each module-level import has to be replicated multiple times
 for each local scope, especially in unittest blocks.  A better number is
 more like 20-30%, as I pointed out 4 out of 13 modules remain at
 top-level in std.array. Using that metric, a 3-4X reduction in top-level
 imports has led to at least a 2.2x improvement in imported files, so the
 effort has been more meaningful than you conclude.


Fixed. I also made a note about the need to duplicate imports as they 
are pushed down.


 Second, as I noted above, most top-level imports have not been made
 selective yet, because of the symbol leak bug that was recently fixed by
 Martin.  You will see in my PRs that I only list those symbols as a
 comment, because I could not turn those into selective imports yet.  If
 the compiler is doing its job right, selective imports should greatly
 reduce the cost of importing a module, even if your metric would still
 show the module being imported.


That is not relevant to this section, which discusses the effectiveness 
of using local imports with the current compilation technology. Per the 
section's opening sentence:


 A legitimate question to ask is whether consistent use of local
 imports wherever possible would be an appropriate approximation of
 the Dependency-Carrying Declarations goal with no change in the
 language at all.


The section "Alternative: Lazy Imports" discusses how static or local 
imports could be used in conjunction with new compilation technologies. 
If there are improvements to be made there, please advise.


 Third, checking some of the output from the commands you ran in your
 script shows that up to half of the imported modules are from druntime.
 I noted earlier that Ilya usually didn't bother scoping top-level
 druntime imports, because he perceived their cost to be low (I scoped
 them too in the handful I cleaned up, just for completeness).  As far as
 I know, nobody has bothered to spend any time scoping druntime, so it
 would be better if you filtered out druntime imports from your analysis.


Fixed to only count imports from std.


 Finally, while it's nice to know the extent of the dependency graph,
 what really matters is the _cost_ of each link of the graph, which is
 what I keep hammering on.  If the cost of links is small, it doesn't
 matter how entangled it is.  If minimizing the dependency graph through
 scoping alone, ie without implementing this DIP, removes most of the
 cost, that's all I care about.


In first approximation, whether a file gets opened or not makes a 
difference (filesystem operations (possibly including networking), 
necessity to rebuild if dependent code is changed. The analysis shows 
there is significant overhead remaining, on average 10.5 additional 
files per unused import.

If the current document could be clearer in explaining costs, please let 
me know.


 I have noted one example above, where _a single DCD in phobos_, ie a
 scoped, selective import, had gigantic costs in terms of executable
 size, where entire modules were included because of it.  If that's the
 case more generally, then _no_ amount of dependency disentangling will
 matter, because the cost of single DCDs is still huge.  Perhaps that's
 just an isolated issue however, it needs to be investigated.


That seems an unrelated matter. Yes, you could pull a large dependency 
in one shot with old or new technology.


 My point is that the dependency graph matters, but now that we're
 getting down to the last entanglements, we need to know the cost of
 those last links.  Your dependency analysis gives us some quantitative
 idea of the size of the remaining graph, but tells us nothing about the
 cost of those links. That's what I'm looking for.

 I will spend some time now investigating those costs with sample code.
 My request all along has been that you give us some idea of those costs,
 if you know the answer already.


I don't know how to make matters much clearer than the current document. 
Any suggestions are welcome. The section "Workaround: Are Local Imports 
Good Enough?" discusses the material cost in terms of extra files that 
need to be opened and parsed (some unnecessarily) in order to complete a 
compilation. The "Rationale" part of the document discusses the costs in 
terms of maintainability, clarity, and documentation.


Thanks,

Andrei
Dec 21 2016
next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Wednesday, 21 December 2016 at 15:16:34 UTC, Andrei 
Alexandrescu wrote:

 On 12/20/2016 11:32 PM, Joakim wrote:

 Second, as I noted above, most top-level imports have not been 
 made
 selective yet, because of the symbol leak bug that was 
 recently fixed by
 Martin.  You will see in my PRs that I only list those symbols 
 as a
 comment, because I could not turn those into selective imports 
 yet.  If
 the compiler is doing its job right, selective imports should 
 greatly
 reduce the cost of importing a module, even if your metric 
 would still
 show the module being imported.


 That is not relevant to this section, which discusses the 
 effectiveness of using local imports with the current 
 compilation technology. Per the section's opening sentence:


 A legitimate question to ask is whether consistent use of local
 imports wherever possible would be an appropriate 
 approximation of
 the Dependency-Carrying Declarations goal with no change in the
 language at all.




It is relevant because it could further reduce the cost from 
module-scope imports.  Whether your section only chooses to focus 
on locally scoped imports and ignore the impact of selective 
imports is irrelevant to me.


 Finally, while it's nice to know the extent of the dependency 
 graph,
 what really matters is the _cost_ of each link of the graph, 
 which is
 what I keep hammering on.  If the cost of links is small, it 
 doesn't
 matter how entangled it is.  If minimizing the dependency 
 graph through
 scoping alone, ie without implementing this DIP, removes most 
 of the
 cost, that's all I care about.


 In first approximation, whether a file gets opened or not makes 
 a difference (filesystem operations (possibly including 
 networking), necessity to rebuild if dependent code is changed. 
 The analysis shows there is significant overhead remaining, on 
 average 10.5 additional files per unused import.


Opening a file or 10 is extremely cheap compared to all the other 
costs of the compiler.  Purely from a technical cost perspective, 
I'm not sure even scoped imports were worth it, as my simple 
investigation below suggests.


 If the current document could be clearer in explaining costs, 
 please let me know.


Yes, please explain what significant "overhead" was reduced by 
scoping imports and would be futher reduced by this DIP.  Opening 
files doesn't cut it.


 My point is that the dependency graph matters, but now that 
 we're
 getting down to the last entanglements, we need to know the 
 cost of
 those last links.  Your dependency analysis gives us some 
 quantitative
 idea of the size of the remaining graph, but tells us nothing 
 about the
 cost of those links. That's what I'm looking for.

 I will spend some time now investigating those costs with 
 sample code.
 My request all along has been that you give us some idea of 
 those costs,
 if you know the answer already.


 I don't know how to make matters much clearer than the current 
 document. Any suggestions are welcome. The section "Workaround: 
 Are Local Imports Good Enough?" discusses the material cost in 
 terms of extra files that need to be opened and parsed (some 
 unnecessarily) in order to complete a compilation. The 
 "Rationale" part of the document discusses the costs in terms 
 of maintainability, clarity, and documentation.


I don't know how to make it clearer that that's not good enough.  
You seem to understand that I want more justification than 
hand-waving about "scalable" and "overhead," which is why you now 
give the cost of opening files as justification, but you don't 
seem to have anything more substantive than that flimsy claim.

I just tried compiling phobos and its unittests for dmd 2.066.1 
and 2.067.1, the dmd releases from right before and right after 
Ilya's PRs linked above (compiling phobos from the older release 
with the newer dmd didn't work and I wasn't interested in porting 
it).  I found they took about the same amount of time to compile 
their respective phobos and the later version consistently took 
15 seconds longer to compile the unittests on a single core.

This suggests that there has been essentially no technical 
benefit to scoped imports, that it is largely superficial.  
That's fine, I think it's worth it just from the standpoint of 
understanding where most dependencies are coming from.  I don't 
think an additional syntax change is necessary for the remaining 
few module-scope dependencies for template constraints.

Note on investigation: Of course, some other relevant factors 
could have changed in dmd and phobos between those two releases, 
so the result is certainly not conclusive.  But the fact that 
there was no decrease in compile times while phobos took around 
the same amount of time is highly suggestive.

I'm uninterested in investigating further given the consistent 
hand-waving justifications for this DIP.  If somebody else had 
submitted this DIP, it would have been quickly shot down, and 
rightly so.
Dec 22 2016
next sibling parent Timothee Cour via Digitalmars-d <digitalmars-d puremagic.com> writes:
 I just tried compiling phobos and its unittests for dmd 2.066.1 and


2.067.1

I think it's worth considering compile time for partial recompilation as
opposed to full compilation. The benifit of this DIP should be more
pronounced there since there'll be more opportunities to skip parsing
modules in that case. Partial recompilation is what matters most during
`edit compile debug cycle` anyways

On Thu, Dec 22, 2016 at 2:17 AM, Joakim via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Wednesday, 21 December 2016 at 15:16:34 UTC, Andrei Alexandrescu wrote:


 On 12/20/2016 11:32 PM, Joakim wrote:


 Second, as I noted above, most top-level imports have not been made
 selective yet, because of the symbol leak bug that was recently fixed by
 Martin.  You will see in my PRs that I only list those symbols as a
 comment, because I could not turn those into selective imports yet.  If
 the compiler is doing its job right, selective imports should greatly
 reduce the cost of importing a module, even if your metric would still
 show the module being imported.


 That is not relevant to this section, which discusses the effectiveness
 of using local imports with the current compilation technology. Per the
 section's opening sentence:

 A legitimate question to ask is whether consistent use of local

 imports wherever possible would be an appropriate approximation of
 the Dependency-Carrying Declarations goal with no change in the
 language at all.




 It is relevant because it could further reduce the cost from module-scope
 imports.  Whether your section only chooses to focus on locally scoped
 imports and ignore the impact of selective imports is irrelevant to me.

 Finally, while it's nice to know the extent of the dependency graph,

 what really matters is the _cost_ of each link of the graph, which is
 what I keep hammering on.  If the cost of links is small, it doesn't
 matter how entangled it is.  If minimizing the dependency graph through
 scoping alone, ie without implementing this DIP, removes most of the
 cost, that's all I care about.


 In first approximation, whether a file gets opened or not makes a
 difference (filesystem operations (possibly including networking),
 necessity to rebuild if dependent code is changed. The analysis shows there
 is significant overhead remaining, on average 10.5 additional files per
 unused import.


 Opening a file or 10 is extremely cheap compared to all the other costs of
 the compiler.  Purely from a technical cost perspective, I'm not sure even
 scoped imports were worth it, as my simple investigation below suggests.

 If the current document could be clearer in explaining costs, please let

 me know.


 Yes, please explain what significant "overhead" was reduced by scoping
 imports and would be futher reduced by this DIP.  Opening files doesn't cut
 it.

 My point is that the dependency graph matters, but now that we're

 getting down to the last entanglements, we need to know the cost of
 those last links.  Your dependency analysis gives us some quantitative
 idea of the size of the remaining graph, but tells us nothing about the
 cost of those links. That's what I'm looking for.

 I will spend some time now investigating those costs with sample code.
 My request all along has been that you give us some idea of those costs,
 if you know the answer already.


 I don't know how to make matters much clearer than the current document.
 Any suggestions are welcome. The section "Workaround: Are Local Imports
 Good Enough?" discusses the material cost in terms of extra files that need
 to be opened and parsed (some unnecessarily) in order to complete a
 compilation. The "Rationale" part of the document discusses the costs in
 terms of maintainability, clarity, and documentation.


 I don't know how to make it clearer that that's not good enough.  You seem
 to understand that I want more justification than hand-waving about
 "scalable" and "overhead," which is why you now give the cost of opening
 files as justification, but you don't seem to have anything more
 substantive than that flimsy claim.

 I just tried compiling phobos and its unittests for dmd 2.066.1 and
 2.067.1, the dmd releases from right before and right after Ilya's PRs
 linked above (compiling phobos from the older release with the newer dmd
 didn't work and I wasn't interested in porting it).  I found they took
 about the same amount of time to compile their respective phobos and the
 later version consistently took 15 seconds longer to compile the unittests
 on a single core.

 This suggests that there has been essentially no technical benefit to
 scoped imports, that it is largely superficial.  That's fine, I think it's
 worth it just from the standpoint of understanding where most dependencies
 are coming from.  I don't think an additional syntax change is necessary
 for the remaining few module-scope dependencies for template constraints.

 Note on investigation: Of course, some other relevant factors could have
 changed in dmd and phobos between those two releases, so the result is
 certainly not conclusive.  But the fact that there was no decrease in
 compile times while phobos took around the same amount of time is highly
 suggestive.

 I'm uninterested in investigating further given the consistent hand-waving
 justifications for this DIP.  If somebody else had submitted this DIP, it
 would have been quickly shot down, and rightly so.
Dec 22 2016
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/22/2016 05:17 AM, Joakim wrote:

 On Wednesday, 21 December 2016 at 15:16:34 UTC, Andrei Alexandrescu wrote:

 In first approximation, whether a file gets opened or not makes a
 difference (filesystem operations (possibly including networking),
 necessity to rebuild if dependent code is changed. The analysis shows
 there is significant overhead remaining, on average 10.5 additional
 files per unused import.


 Opening a file or 10 is extremely cheap compared to all the other costs
 of the compiler.  Purely from a technical cost perspective, I'm not sure
 even scoped imports were worth it, as my simple investigation below
 suggests.


This is a misunderstanding. (I'll make the DIP clearer.) It's not about 
the cost of opening the file per se; I'm using the number of files 
opened as a proxy for all work involved in processing a file. Meaning, 
if you import 10 files you're likely to do roughly 10x the work of 
importing one file.


 I just tried compiling phobos and its unittests for dmd 2.066.1 and
 2.067.1, the dmd releases from right before and right after Ilya's PRs
 linked above (compiling phobos from the older release with the newer dmd
 didn't work and I wasn't interested in porting it).  I found they took
 about the same amount of time to compile their respective phobos and the
 later version consistently took 15 seconds longer to compile the
 unittests on a single core.


Unittesting Phobos (whether in separation or together) will instantiate 
everything so they are the case when local imports do _not_ make any 
difference. That's why DIP1005 uses that case as an estimate of 
everything in an imported module being used.

Compiling Phobos in its entirety (one command) and measuring that time 
is also of tenuous relevance; I need to think of it but at first sight 
the cost of unnecessary imports is collapsed, and I suspect (and will 
measure) that either way most modules are in fact imported.

One measurement of interest is: write a module that imports exactly one 
stdlib module, compile it, and measure time. Looking at the generated 
object and executable size would be also of interest. I'll do that.


 This suggests that there has been essentially no technical benefit to
 scoped imports, that it is largely superficial.  That's fine, I think
 it's worth it just from the standpoint of understanding where most
 dependencies are coming from.  I don't think an additional syntax change
 is necessary for the remaining few module-scope dependencies for
 template constraints.

 Note on investigation: Of course, some other relevant factors could have
 changed in dmd and phobos between those two releases, so the result is
 certainly not conclusive.  But the fact that there was no decrease in
 compile times while phobos took around the same amount of time is highly
 suggestive.

 I'm uninterested in investigating further given the consistent
 hand-waving justifications for this DIP.  If somebody else had submitted
 this DIP, it would have been quickly shot down, and rightly so.


Walter and I have the role of scrutinizing every addition to the 
language (and reject most). It is natural that our own work is met with 
increased scrutiny. Picking to death anything and everything we say or 
do is a staple in this community, and a rite of passage on github. It is 
of course impossible to know what would have happened if the proposal 
were made by someone else. All I can say is Walter knew nothing about it 
and said it is good (except for the initial syntax; he's on board with 
"with").

Anyhow, not to worry. The burden of proof is on the DIP. I'll take a 
look at making some more measurements.


Andrei
Dec 22 2016
prev sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Thu, 22 Dec 2016 10:17:33 +0000, Joakim wrote:

 Opening a file or 10 is extremely cheap compared to all the other costs
 of the compiler.  Purely from a technical cost perspective,
 I'm not sure even scoped imports were worth it, as my simple
 investigation below suggests.


The compiler doesn't merely have to open the file. It has to run at least 
partial semantic analysis on it in order to locate symbols. When 
templates are involved, this can become quite expensive.

With static and selective imports, this step can be entirely avoided -- 
the current file tells you which files contain which declarations. With 
either, the compiler can tell exactly which file contains which 
declarations, so it can avoid pulling in anything beyond what's strictly 
necessary.

With declaration-scoped imports, you avoid the same step. However, it 
requires the maintainer's discipline to reduce the declaration's imports 
to the minimal possible set of imports.

For instance, I have a declaration:

  import myapp.users, std.socket;
  bool isUserOnline(User user, Socket userSocket);

I decide that this needs compilation time optimizations. The current way:

  static import myapp.users, std.socket;
  bool isUserOnline(myapp.users.User user, std.socket.Socket socket);

And Andrei's way:

  bool isUserOnline(User user, Socket userSocket)
  import myapp.users, std.socket;

I refactor things so that this check finds the user socket on its own. 
The current way:

  static import myapp.users, std.socket;
  bool isUserOnline(myapp.users.User user);

Eh, I forgot I don't need std.socket anymore, but this costs a few 
microseconds of compiler time to add it to the current symbol table. It 
has to allocate a lazily expanded module stub. Shouldn't be a huge deal.

Andrei's way:

  bool isUserOnline(User user)
  import myapp.users, std.socket;

Again, I forgot to update the imports, but this time the compiler has to 
read std.socket from disk, parse it, run semantic on it, and import all 
its top-level symbols into the scope's symbol table. Because there's 
nothing here that says the 'User' type is in myapp.users instead of 
std.socket.

---

In point of fact, selective and static imports should be *faster* than 
Andrei's way. Consider:

  static import myapp.users, std.socket;
  bool isUserOnline(myapp.users.User user, std.socket.Socket socket);

This has to locate a declaration named `User` in myapp.users, and it has 
to locate a declaration named `Socket` in std.socket.

But let's look at Andrei's way:

  bool isUserOnline(User user, Socket userSocket)
  import myapp.users, std.socket;

Here, the compiler has to search *both* myapp.users and std.socket for a 
declaration named `User`, then it has to search both for `Socket`. (Even 
if it finds `User` in the first, it still needs to search the second in 
case both define that symbol.)

You go from O(distinct number of types referenced) lookups to O(types * 
imports).

Granted, you'll usually have between one and three types, between one and 
three imports, so the point is a bit less salient.
Dec 22 2016
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/22/16 9:53 PM, Chris Wright wrote:

   bool isUserOnline(User user, Socket userSocket)
   import myapp.users, std.socket;


Has changed to

with (import myapp.users, std.socket)
bool isUserOnline(User user, Socket userSocket);

Andrei
Dec 23 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/22/16 9:53 PM, Chris Wright wrote:

 In point of fact, selective and static imports should be *faster* than
 Andrei's way. Consider:

   static import myapp.users, std.socket;
   bool isUserOnline(myapp.users.User user, std.socket.Socket socket);

 This has to locate a declaration named `User` in myapp.users, and it has
 to locate a declaration named `Socket` in std.socket.

 But let's look at Andrei's way:

   bool isUserOnline(User user, Socket userSocket)
   import myapp.users, std.socket;


with (static import myapp.users, std.socket)
bool isUserOnline(myapp.users.User user, std.socket.Socket socket);


Andrei
Dec 23 2016
prev sibling parent piotrklos <poliklosio happypizza.com> writes:
On Wednesday, 21 December 2016 at 15:16:34 UTC, Andrei 
Alexandrescu wrote:

 On 12/20/2016 11:32 PM, Joakim wrote:

 (...)


 I don't know how to make matters much clearer than the current 
 document. Any suggestions are welcome. The section "Workaround: 
 Are Local Imports Good Enough?" discusses the material cost in 
 terms of extra files that need to be opened and parsed (some 
 unnecessarily) in order to complete a compilation. The 
 "Rationale" part of the document discusses the costs in terms 
 of maintainability, clarity, and documentation.


 Thanks,

 Andrei


Stipulation: I think the difference of opinion may be caused by 
working on different sizes of projects in ones career (tens of 
thousands vs millions of LoC).

Suggestion 1: Maybe the DIP should point out that the cost of 
redundant imports (however small) tends to grow quadratically 
with code size (size of import tree times the number of 
compilations).
If this is not the case then maybe the DIP is really in the wrong 
direction.

Suggestion 2:
Implement the DIP, autogenerate millions of lines of D code (2 
versions: with and without DCDs) and see which version of DMD 
compiles them faster. This may also expose other ways to improve 
scalability without implementing this DIP.
Dec 22 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/20/2016 03:46 AM, Joakim wrote:

 I didn't just say "eh:" I gave evidence for why I think the problem is
 minimal and asked why it's so important to scope those last 3-4 imported
 modules too, which you didn't answer.


You have asked for a smoking gun, and one has been found.

I have just uploaded a major update that carefully analyzes the 
improvements brought about by switching to local imports in the D 
Standard Library. Please refer to the section "Workaround: Are Local 
Imports Good Enough?" and Appendix A:

https://github.com/dlang/DIPs/pull/51/files

https://github.com/dlang/DIPs/blob/91baecedcfe7cb75ac22e66478722ec797ebb901/DIPs/DIP1005.md


Andrei
Dec 20 2016
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/20/2016 05:08 PM, Andrei Alexandrescu wrote:

 On 12/20/2016 03:46 AM, Joakim wrote:

 I didn't just say "eh:" I gave evidence for why I think the problem is
 minimal and asked why it's so important to scope those last 3-4 imported
 modules too, which you didn't answer.


 You have asked for a smoking gun, and one has been found.

 I have just uploaded a major update that carefully analyzes the
 improvements brought about by switching to local imports in the D
 Standard Library. Please refer to the section "Workaround: Are Local
 Imports Good Enough?" and Appendix A:

 https://github.com/dlang/DIPs/pull/51/files

 https://github.com/dlang/DIPs/blob/91baecedcfe7cb75ac22e66478722ec797ebb901/DIPs/DIP1005.md


I've also added a supporting script that creates the table in Appendix 
A: 
https://github.com/andralex/DIPs/blob/9e9ebc8738ce04b2d85a92feafacc7ef81e59811/DIPs/DI
1005-countlines.zsh 
-- Andrei
Dec 20 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/20/2016 05:08 PM, Andrei Alexandrescu wrote:

 On 12/20/2016 03:46 AM, Joakim wrote:

 I didn't just say "eh:" I gave evidence for why I think the problem is
 minimal and asked why it's so important to scope those last 3-4 imported
 modules too, which you didn't answer.


 You have asked for a smoking gun, and one has been found.

 I have just uploaded a major update that carefully analyzes the
 improvements brought about by switching to local imports in the D
 Standard Library. Please refer to the section "Workaround: Are Local
 Imports Good Enough?" and Appendix A:

 https://github.com/dlang/DIPs/pull/51/files

 https://github.com/dlang/DIPs/blob/91baecedcfe7cb75ac22e66478722ec797ebb901/DIPs/DIP1005.md


Eh, looks like http://dillinger.io/ and github don't agree on table 
rendering... Fixed URL for nice viewing:

https://github.com/dlang/DIPs/blob/249b28ddd784220e44e343e78e5ea7a65c4c7bde/DIPs/DIP1005.md


Andrei
Dec 20 2016
prev sibling parent reply Timothee Cour via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, Dec 19, 2016 at 9:33 PM, Timothee Cour <thelastmammoth gmail.com>
wrote:


 what about using `lazy` instead of `with`:

 `with(import foo)`
 =>
 `lazy(import foo)`

 advantages:
 * avoids confusion regarding usual scoping rules of `with` ;
 * conveys that the import is indeed lazy

 Furthermore (regardless of which keyword is used), what about allowing `:`
 ```
 // case 1
 lazy(import foo)
 void fun(){}

 // case 2
 lazy(import foo) {
   void fun(){}
 }

 // case 3 : this is new
 lazy(import foo):
 void fun1(){}
 void fun2(){}
 ```

 advantages:

 * same behavior as other constructs which don't introduce a scope:
 ```
 // case 1, 2 3 are allowed:
 version(A):
 static if(true):
 private:
 void fun(){}
 ```

 * avoids nesting when case 3 is used (compared to when using `{}`)

 * I would argue that grouping lazy imports is actually a common case;
 without case 3, the indentation will increase.



Andrei: ping on this? (especially regarding allowing `:`)
Dec 21 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/21/16 6:40 PM, Timothee Cour via Digitalmars-d wrote:

 Andrei: ping on this? (especially regarding allowing `:`)


I think "lazy" is a bit too cute. "with" is so close to what's actually 
needed, it would be a waste to not use it.

Generally I'm weary of the use of ":" (never liked it - it makes code 
dependent on long-distance context) so I'd rather snatch the opportunity 
to avoid it.


Andrei
Dec 21 2016
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/13/2016 05:33 PM, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


Major changes effected to the DIP, which affect it. In-depth discussion 
of lazy imports, a change in lookup rules, and a few minor things:

https://github.com/dlang/DIPs/blob/d228f48a12c1d976735412fd5b7543cb1febda7b/DIPs/DIP1005.md


Andrei
Dec 16 2016
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2016-12-13 23:33, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


A couple of questions.

1. The text says:

"void process(File input) import (std.stdio);

With this syntax, the import is executed only if the declared name is 
actually looked up."

Is "declared name" referring to the function "process"? To me it sounds 
like it could be misinterpreted to refer to "File", which would indicate 
the import is lazily executed.

2. The text says:

"Currently inline imports do not apply to alias declarations, enumerated 
types declarations, or variable declarations"

What's the reasoning behind this?

3. Is it required to place the import after the declaration or can it be 
placed in front of it?

4. Related to the third question. Would it be better (if not already) to 
implement the import as an attribute, which would allow for a more 
flexible syntax:

void process(File input) import (std.stdio);
import (std.stdio) void process(File input);

import (std.stdio)
{
     void process(File input);
     void process2(File input);
}

import (std.stdio):

void process(File input);
void process2(File input);

5. About the original syntax that was used, i.e. 
"import.std.range.isInputRange!Range". That syntax could require to 
always use a colon to separate the module name from the imported symbol. 
It's the same syntax as a selective import, which I think fits since 
with this syntax it will always be selective imports, as far as I 
understand. Example:

void process(import.std.stdio:File input);

This syntax could also be extended to all symbols:

void process(import.std.stdio:File input)
{
     import.std.stdio:File a = input;
}

6. Not sure if it's a requirement for a DIP but I don't see any formal 
description of the necessary grammar changes.

-- 
/Jacob Carlborg
Dec 17 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/17/2016 10:39 AM, Jacob Carlborg wrote:

 On 2016-12-13 23:33, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 A couple of questions.

 1. The text says:

 "void process(File input) import (std.stdio);

 With this syntax, the import is executed only if the declared name is
 actually looked up."

 Is "declared name" referring to the function "process"? To me it sounds
 like it could be misinterpreted to refer to "File", which would indicate
 the import is lazily executed.


[snip]

Thanks, made a change. -- Andrei
Dec 19 2016
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Pushed again, now with the syntax using "with" proposed by Hatem Oraby 
and others.

https://github.com/dlang/DIPs/pull/51

https://github.com/dlang/DIPs/blob/71bde077488b566fba7603de6095b45984d9294a/DIPs/DIP1005.md


Andrei
Dec 19 2016
parent Meta <jared771 gmail.com> writes:
On Tuesday, 20 December 2016 at 01:06:01 UTC, Andrei Alexandrescu 
wrote:

 Pushed again, now with the syntax using "with" proposed by 
 Hatem Oraby and others.

 https://github.com/dlang/DIPs/pull/51

 https://github.com/dlang/DIPs/blob/71bde077488b566fba7603de6095b45984d9294a/DIPs/DIP1005.md


 Andrei


"In addition, we propose the statement and declaration with 
(import ImportList). ImportList is any syntactical construct 
currently accepted by the import declaration. The with (import 
ImportList) obeys the following rules:

- Inside any function, with (Import ImportList) is a statement 
that introduces a scope. Inside the with, lookup considers the 
import local to the declaration (similar to the current handling 
of nested imports).
- Everywhere else, with (Import ImportList) is always a 
declaration and does not introduce a new scope. Lookup of symbols 
is the same as for the statement case."

I must've somehow missed this when reading through for the first 
time after your changes as I thought this was a glaring omission. 
Glad to see that this was covered as this rightly makes it 
turtles all the way down. Actually, you could completely replace 
import statements with this new construct, having the `import 
std.range` at the top level replaced with a module-wide attribute 
`with (import std.range):`. In that light my only real criticism 
is that it's kind of redundant having both this new form *and* 
local imports, and it's not clear which is better and why to a 
new user.
Dec 20 2016
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Added this note about module constructors:



Inline and scoped imports offer the option of better handling of static 
module constructors. Currently, modules that mutually `import` one 
another (either directly or through a longer chain) cannot 
simultaneously define `shared static this()` constructors. The reason is 
that, again, dependencies are computed at module level.

If instead modules have no top-level dependencies, then the compiler is 
able to compute the narrow set of dependencies needed for executing the 
static module constructor. The static constructor may be (a) a part of a 
`with` declaration, (b) use local imports within, and (c) call other 
functions within the module that have their own dependencies. For example:

```d
// assume no top-level import
with (module_a) void fun(T)()
{
     import module_b;
     return gun();
}
with (module_c)
static shared this()
{
     import module_d;
     fun!int;
}
```

In this case, the module constructor depends (only) on `module_a`, 
`module_b`, `module_c`, and `module_d`. The full information is confined 
within the current module so it is inferrable during separate compilation.



Andrei
Dec 19 2016
prev sibling next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 12/13/16 11:33 PM, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


Just a thought but with all of proliferation of imports down to each 
declaration comes the pain that e.g. renaming a module cascades to 
countless instances of import statements. This is true of local imports 
as well but the problem gets bigger.

----
Dmitry Olshansky
Dec 20 2016
next sibling parent Meta <jared771 gmail.com> writes:
On Tuesday, 20 December 2016 at 14:31:38 UTC, Dmitry Olshansky 
wrote:

 On 12/13/16 11:33 PM, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


 Just a thought but with all of proliferation of imports down to 
 each declaration comes the pain that e.g. renaming a module 
 cascades to countless instances of import statements. This is 
 true of local imports as well but the problem gets bigger.

 ----
 Dmitry Olshansky


Could you not have the old module just be empty and publicly 
import the new one, and also deprecate it so people have time to 
change their code? Also having special syntax makes finding every 
occurrence of an inline import a fairly simple search.
Dec 20 2016
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/20/2016 09:31 AM, Dmitry Olshansky wrote:

 On 12/13/16 11:33 PM, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


 Just a thought but with all of proliferation of imports down to each
 declaration comes the pain that e.g. renaming a module cascades to
 countless instances of import statements. This is true of local imports
 as well but the problem gets bigger.


Good point, I'll integrate it in the document. -- Andrei
Dec 20 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/20/2016 09:31 AM, Dmitry Olshansky wrote:

 On 12/13/16 11:33 PM, Andrei Alexandrescu wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei


 Just a thought but with all of proliferation of imports down to each
 declaration comes the pain that e.g. renaming a module cascades to
 countless instances of import statements. This is true of local imports
 as well but the problem gets bigger.


https://github.com/dlang/DIPs/pull/51/commits/d4ef6826dacedc38f822e48bec2186d93040fb42

Andrei
Dec 21 2016
prev sibling next sibling parent reply Chris Wright <dhasenan gmail.com> writes:
I'm looking at this part:


 The manual conversion of std.array to the "static import" form is shown
 here. It leads to the expected lengthening of the symbols used in
 declarations, which appears to eliminate one disadvantage by
 introducing another.


You get longer declarations when you're reading and writing code. This 
shouldn't impact the docs. Not arguing that there is a disadvantage, just 
pointing out the scope is relatively small.

All your references are entirely unambiguous. If I'm reading the source 
and I'm unfamiliar with phobos, I don't have to wonder if that template 
constraint comes from std.string or std.traits.

It's two benefits for one disadvantage.

You can also have renamed imports to reduce the amount of typing:

  static import trait = std.traits;
  trait.ForeachType!Range[] array(Range)(Range r)
  if (trait.isIterable!Range &&
      trait.isNarrowString!Range &&
      !trait.isInfinite!Range)) {
  }


 The manual conversion of std.array to the "selective import" form is 
 shown here. Conversion was successful but because it collapses all 
 imports at the top, it does not make it much easier to identify e.g. 
 what dependencies would be pulled if a given artifact in std.array were 
 used.


It's still trivial for the compiler to detect which modules a particular 
declaration depends on.

If I'm in vim, I can put my cursor on the declaration, hit '*', navigate 
to the top of the document, and find the next match. It's like five 
keypresses. With VS Code or the like, I can similarly go to the top of 
the file and find the first match without much trouble.

And this process tells me which module defines the symbol. Either the 
first hit is at the top of the file in the import section, and I know 
which file to look in, or it's not, and I know it's defined locally (and 
there's a good chance I'm at the declaration).

With your proposal, when I'm lucky, I know the declaration must be either 
in the current file or in the locally imported file. (When I'm unlucky, 
the file has some top-level imports or multiple local imports.) And since 
the utility of this is mostly for large modules, I'm still going to pull 
out grep. It *does* help when I need to import the same thing elsewhere, 
at least.


 Again the manual process was highly nontrivial.


This is also true for your proposal.

CyberShadow also offered to whip up a tool to automate the conversion to 
selective imports. Presumably they would be able to do something similar 
with your proposal.

So this is kind of irrelevant.
Dec 22 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/22/16 10:31 PM, Chris Wright wrote:

 It's two benefits for one disadvantage.


One won't make decisions by taking the difference of pros and cons, 
right? -- Andrei
Dec 23 2016
parent Chris Wright <dhasenan gmail.com> writes:
On Fri, 23 Dec 2016 07:48:55 -0500, Andrei Alexandrescu wrote:


 On 12/22/16 10:31 PM, Chris Wright wrote:

 It's two benefits for one disadvantage.


 
 One won't make decisions by taking the difference of pros and cons,
 right? -- Andrei


You'd use the weighted difference, naturally.
Dec 23 2016
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Major update adding an experiment that shows the cost of top-level imports.

https://github.com/dlang/DIPs/pull/51

https://github.com/dlang/DIPs/blob/a3ef4e25cfb9f884fee29edb5553a3a2b840f679/DIPs/DIP1005.md

Relevant added text:

Another matter we investigated is how reducing top-level imports 
influences build times and the size of the object files produced. We do 
not have an experimental implementation of this DIP, so measuring impact 
directly was not possible. We did the converse experiment---adding 
top-level imports.

In a separate branch of the standard library code, for each module we 
added all nested imports back to the top level. Some hand-editing was 
needed after that because of clashes in symbol names. Also, some imports 
needed to be removed because of circular dependencies and related 
limitations in the language's design and implementation. The resulting 
setup can be seen in 
[PR4992](https://github.com/dlang/phobos/pull/4992). Then for each 
module in the standard library we compiled one file that consists of 
exactly one `import` declaration, monitoring compile time and object 
file size. Appendix B displays build times and size of object files 
produced by this experiment.

|Aggregate|Time (top-level)|Time (nested)|Object size (top-level)|Object 
size (nested)|
|---|---|---|---|---|
|Median|320ms|13788|32ms|4296|
|Average|287.6ms|13437.2|64.6ms|5734.1|

As expected, the experiment shows that both build times and object file 
sizes were improved by moving imports away from the top level. We 
estimate that eliminating the 10.5x slack dependency fan-in will bring 
import costs down to negligible and also bring object file size down.


Andrei
Dec 23 2016
parent reply Chris Wright <dhasenan gmail.com> writes:
On Fri, 23 Dec 2016 11:25:41 -0500, Andrei Alexandrescu wrote:

 Dependency-Carrying Declarations allow scalable template libraries.



 template libraries


*This* is the part I hadn't noticed before, and it explains the confusion 
I had.

If you *only* apply this to templates, it works. The current situation is 
a result of where template constraints are located. If there were a 
`static throw` equivalent for indicating that parameters were invalid, or 
if template constraints were in an `in` block like contracts, then this 
wouldn't even have come up. If phobos went for object oriented code 
instead of templates (as an example, not a recommendation), then this 
wouldn't be an issue.

However, at that point, it would be utterly useless to me. I'm looking at 
my entire dub package cache, plus the ten-ish most recently updated dub 
packages:

* units-d uses allSatisfy. Once.
* vibe has two structs that would benefit, except they're inside a 
unittest. I've never compiled dub's unittests.

Template constraints have little adoption outside phobos. When they *are* 
used, they tend to use language facilities instead of templates to 
express the condition. And when a template is used, it tends to be 
defined in the same module where it's used.

I grant that everyone uses phobos, and phobos uses template constraints a 
lot. If it's *just* a problem inside phobos, though, there's another 
obvious solution: split up modules that tend to house a lot of template 
constraints. Split up the modules that use a wide variety of template 
constraints.

Then I can decide whether the convenience of not hunting for narrower 
imports is worth the extra quarter second of compilation.

----

Now, if you want to apply this to things that are *not* templates, then 
you could get a lot further. However, you would end up with code that 
compiles when it wouldn't today. Code that compiles because the portions 
you use would compile if they were on their own, while other bits 
wouldn't. That's the status quo for templates, even no-arg templates, but 
a change from what we currently do everywhere else.

And *that* is what would make it equivalent to use static or selective 
imports. It would also increase the utility from my perspective from "why 
the hell are we even doing this?" to "that's kinda nice".
Dec 23 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/23/2016 05:33 PM, Chris Wright wrote:

 On Fri, 23 Dec 2016 11:25:41 -0500, Andrei Alexandrescu wrote:

 Dependency-Carrying Declarations allow scalable template libraries.



 template libraries


 *This* is the part I hadn't noticed before, and it explains the confusion
 I had.

 If you *only* apply this to templates, it works. The current situation is
 a result of where template constraints are located. If there were a
 `static throw` equivalent for indicating that parameters were invalid, or
 if template constraints were in an `in` block like contracts, then this
 wouldn't even have come up. If phobos went for object oriented code
 instead of templates (as an example, not a recommendation), then this
 wouldn't be an issue.


I acknowledge that if the language were defined a different way this 
issue wouldn't have come up. But this is a truism - one can say that 
about any issue.


 However, at that point, it would be utterly useless to me. I'm looking at
 my entire dub package cache, plus the ten-ish most recently updated dub
 packages:

 * units-d uses allSatisfy. Once.
 * vibe has two structs that would benefit, except they're inside a
 unittest. I've never compiled dub's unittests.


Fair enough. I reckon a number of traditional ways of designing software 
would not be helped radically by DIP1005.


 Template constraints have little adoption outside phobos.


That will change. We definitely need to do all we can to support and 
improve language support for template constraints.


 When they *are*
 used, they tend to use language facilities instead of templates to
 express the condition. And when a template is used, it tends to be
 defined in the same module where it's used.


That may be true for some code today but not for future code. std.traits 
gets larger and better with more interesting introspection capabilities. 
I envision introspection as a core differentiating feature of D that 
will put it ahead of all other languages.


 I grant that everyone uses phobos, and phobos uses template constraints a
 lot. If it's *just* a problem inside phobos, though, there's another
 obvious solution: split up modules that tend to house a lot of template
 constraints. Split up the modules that use a wide variety of template
 constraints.


This point is discussed carefully in DIP1005. Please let me know if 
anything needs to be added.


 Now, if you want to apply this to things that are *not* templates, then
 you could get a lot further. However, you would end up with code that
 compiles when it wouldn't today. Code that compiles because the portions
 you use would compile if they were on their own, while other bits
 wouldn't. That's the status quo for templates, even no-arg templates, but
 a change from what we currently do everywhere else.

 And *that* is what would make it equivalent to use static or selective
 imports. It would also increase the utility from my perspective from "why
 the hell are we even doing this?" to "that's kinda nice".


Lazier compilation is indeed a projected benefit of this DIP. I did not 
want to dilute the thrust of the proposal with a remote promise.


Andrei
Dec 23 2016
parent reply Chris Wright <dhasenan gmail.com> writes:
tldr: the performance impact of this DIP would be too small to easily 
measure and only impacts 26 declarations referencing 14 templates in the 
standard library anyway.

On Fri, 23 Dec 2016 18:55:25 -0500, Andrei Alexandrescu wrote:

 I grant that everyone uses phobos, and phobos uses template constraints
 a lot. If it's *just* a problem inside phobos, though, there's another
 obvious solution: split up modules that tend to house a lot of template
 constraints. Split up the modules that use a wide variety of template
 constraints.


 
 This point is discussed carefully in DIP1005. Please let me know if
 anything needs to be added.


An estimate for the actual impact on phobos, since that's your primary 
driver for the change -- both under the status quo and if we try to split 
modules.

The comparison to mach.d is a strawman. When I thought this might be a 
problem within phobos, I thought we'd probably split std.traits and maybe 
std.meta up, probably into 2-5 modules each. Not 150 lines per module; 
more like 1500 to 4000 lines per module.

Then I looked at the code.

Phobos has 26 templates that would use this special syntax, referencing 
14 distinct templates and CTFE functions in three modules.

If you kept the same ratios as are found in mach.d, you'd have one file 
for every template used as a constraint outside its own module, one for 
everything else, and as many files again with nothing in them.

The templates that have to be parsed are std.traits, std.meta, and 
std.range.primitives. I wrote a test program that imported them without 
using them. It compiled in too little time to accurately measure -- 
`time` reports 0.01s wall time.

You want to add new language syntax to save me ten milliseconds during 
compilation.

You hope that template constraints become more common. If they do, this 
change relies on:

* projects defining their own constraints (not just using std.traits)
* those constraints being defined in modules other than where they're used
* those constraints being defined in very large modules
* my code not depending on anything in those constraint-defining modules
* my code depending on something in a file with a template with one of 
those constraints
* this whole situation being common enough to give me a death by a 
thousand papercuts

If any project aside from the standard library has a 7k line module 
defining things mainly used in template constraints, something is 
seriously weird. But on the plus side, it would only cost me 10 
milliseconds. Now, if *dozens* of projects did that, well, I'd be running 
to the relative simplicity of Java APIs long before I worried about 
compilation speed.


 Lazier compilation is indeed a projected benefit of this DIP. I did not
 want to dilute the thrust of the proposal with a remote promise.


Lazier compilation would *obviate* this DIP. Lazy compilation of 
selective and static imports would not require any parser changes and 
would make a lot of code faster (at the cost of allowing some things that 
don't compile to start compiling, as does your proposal). You can't get 
any performance advantages outside templates without implementing lazy 
imports.



Appendix A: templates that would receive 'with(import foo)' in Phobos.

I used a relatively simple regex to look for this. If someone put more 
than one line between a template and its constraints, or comments with 
certain formatting, I may have missed it. However, that would violate the 
phobos style guide.

This doesn't include templates that use constraints defined in their own 
modules, because that module must already be parsed and no efficiency 
gains could be realized.

It also omits a few cases where the module has a strong reason to import 
the dependency aside from template constraints. I believe this was only 
two constraints defined in std.digest, used in one or two other modules.

algorithm/comparison.d:98:template among(values...)
experimental/typecons.d:82:private template implementsInterface(Source, 
Targets...)
experimental/typecons.d:94:    template implementsInterface()
experimental/typecons.d:126:private template implementsInterface(Source, 
Targets...)
experimental/typecons.d:184:template wrap(Targets...)
experimental/typecons.d:237:        template wrap(Source)
range/package.d:2069:template Take(R)
range/package.d:3501:template Cycle(R)
range/interfaces.d:277:template MostDerivedInputRange(R)
range/interfaces.d:336:template InputRangeObject(R)
numeric.d:678:template FPTemporary(F)
conv.d:3894:template octal(alias decimalInteger)
utf.d:1136:package template codeUnitLimit(S)
typecons.d:1779:private mixin template RebindableCommon(T, U, alias This)
typecons.d:1838:template Rebindable(T)
typecons.d:4239:template wrap(Targets...)
typecons.d:4252:    template wrap(Source)
typecons.d:4412:template wrap(Targets...)
typecons.d:4429:template unwrap(Target)
typecons.d:4461:template unwrap(Target)
format.d:657:template FormatSpec(Char)
algorithm/iteration.d:1055:template filter(alias predicate) if (is(typeof
(unaryFun!predicate)))
internal/math/biguintcore.d:81:template maxBigDigits(T) if (isIntegral!T)
meta.d:248:package template OldAlias(T) if (!isAggregateType!T || is
(Unqual!T == T))
meta.d:254:package template OldAlias(T) if (isAggregateType!T && !is
(Unqual!T == T))
utf.d:3542:template byUTF(C) if (isSomeChar!C)


Appendix B: templates that would need to be extracted out in phobos, if 
parsing their modules cost a non-negligible amount of time.

std.meta:
    allSatisfy
    anySatisfy
    ApplyLeft

std.range.primitives:
    hasSlicing
    isInputRange
    isInfinite

std.traits:
    isAggregateType
    isAssociativeArray
    isDynamicArray
    isFloatingPoint
    isImplicitlyConvertible
    isIntegral
    isMutable
    isSomeChar
Dec 23 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/23/16 9:23 PM, Chris Wright wrote:

 The comparison to mach.d is a strawman.


The mach.d is given as an example of the approach of breaking code into 
fine-grained module. No comparison is made or implied.


 Then I looked at the code.

 Phobos has 26 templates that would use this special syntax, referencing
 14 distinct templates and CTFE functions in three modules.


Could you please give more detail about the method you used? What 
special syntax are you referring to - would that be the "with (import 
...)" syntax? If that's the case there's a bunch of stuff missing. 
Consider e.g.:

int cmp(alias pred = "a < b", R1, R2)(R1 r1, R2 r2)
if (isInputRange!R1 && isInputRange!R2 && !(isSomeString!R1 && 
isSomeString!R2));

int cmp(alias pred = "a < b", R1, R2)(R1 r1, R2 r2)
if (isSomeString!R1 && isSomeString!R2);

These and many like them would use the "with import" construct, wouldn't 
they? They are missing from your list.


 The templates that have to be parsed are std.traits, std.meta, and
 std.range.primitives.


That's where most introspection primitives for the standard library are 
situated indeed.


 I wrote a test program that imported them without
 using them. It compiled in too little time to accurately measure --
 `time` reports 0.01s wall time.

 You want to add new language syntax to save me ten milliseconds during
 compilation.


DIP1005 enumerates several benefits of the proposed feature. Speed is 
not the most important and is not presented as such, but it takes most 
space because the others are more difficult to experiment with without 
an experimental implementation.


 You hope that template constraints become more common.


Well, to the extent that is the way correct code is written, that's more 
than just hope.


 If they do, this
 change relies on:

 * projects defining their own constraints (not just using std.traits)
 * those constraints being defined in modules other than where they're used
 * those constraints being defined in very large modules
 * my code not depending on anything in those constraint-defining modules
 * my code depending on something in a file with a template with one of
 those constraints
 * this whole situation being common enough to give me a death by a
 thousand papercuts


It's not only constraints, it's everything in declarations that relies 
on entities (e.g. types) defined in other modules. And again the 
feature's benefits go well beyond making general projects faster to build.

A simple way to look at things is - local imports are The Right Thing, 
for many reasons beyond compilation speed. There needs to be a way to do 
the right thing for the declaration part as well.

(I'm glad we're getting to the point of diminished returns - I recall an 
argument made a while ago was that, on the contrary, our compilation 
model cannot possibly scale for large projects.)


 Appendix A: templates that would receive 'with(import foo)' in Phobos.


Could you please provide more detail on how you measured this and what 
assumptions you made? Most templates in Phobos would receive a with 
clause, so there's a disconnect somewhere.


Andrei
Dec 23 2016
parent Chris Wright <dhasenan gmail.com> writes:
On Fri, 23 Dec 2016 23:41:46 -0500, Andrei Alexandrescu wrote:


 On 12/23/16 9:23 PM, Chris Wright wrote:

 The comparison to mach.d is a strawman.


 
 The mach.d is given as an example of the approach of breaking code into
 fine-grained module. No comparison is made or implied.


"Assuming module size is a project invariant, the number of files scales 
roughly with project size. This means mach.d would need 2000 files to 
scale up to the same size as the D standard library"

That's a direct comparison that you made. You might want to remove that 
from the proposal if you don't want to compare them.


 Could you please give more detail about the method you used?


An insufficient one.

(I would think this sort of analysis would generally be the burden of the 
proposer, generally.)

https://gist.github.com/dhasenan/681b5178672556aa0e5ec8fb4c9eae7e uses 
dmd's json output instead of regular expressions and should be rather 
better.

This shows a total of 35 templates used in template constraints, 124 
usages in total.

The quick usage breakdown:

 1 std.experimental.ndslice.internal
 1 std.experimental.ndslice.slice
24 std.meta
31 std.range.primitives
66 std.traits
 1 std.typecons

Notable mentions: the ndslice templates are used only in other ndslice 
modules, and the std.typecons reference is from 
std.experimental.typecons. So it's still ~10ms of overhead, by both our 
measurements.


 And again the
 feature's benefits go well beyond making general projects faster to
 build.


You may as well remove the parts about compilation efficiency from the 
proposal entirely, then.

The DIP's points here aren't invalid. However, static and selective 
imports work nearly as well. A doc generator that turns referenced 
identifiers into links (like dpldocs.info) works nearly as well for 
reading. For moving files between modules, that's relatively rare, and 
fixing imports is a small part of it. (In the worst case, I can copy all 
imports over and use dustmite to reduce them. Or just leave them.)

It provides small occasional benefits for a nontrivial maintenance cost. 
At least it doesn't mandate that I use it.


 A simple way to look at things is - local imports are The Right Thing,
 for many reasons beyond compilation speed. There needs to be a way to do
 the right thing for the declaration part as well.


Function-scoped imports are a good way to hide implementation details 
from people who don't need them.
Dec 24 2016
prev sibling next sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Saturday, 24 December 2016 at 02:23:53 UTC, Chris Wright wrote:

 Appendix A: templates that would receive 'with(import foo)' in 
 Phobos.

 I used a relatively simple regex to look for this. If someone 
 put more than one line between a template and its constraints, 
 or comments with certain formatting, I may have missed it. 
 However, that would violate the phobos style guide.

 This doesn't include templates that use constraints defined in 
 their own modules, because that module must already be parsed 
 and no efficiency gains could be realized.

 It also omits a few cases where the module has a strong reason 
 to import the dependency aside from template constraints. I 
 believe this was only two constraints defined in std.digest, 
 used in one or two other modules.

 algorithm/comparison.d:98:template among(values...)
 experimental/typecons.d:82:private template 
 implementsInterface(Source,
 Targets...)
 experimental/typecons.d:94:    template implementsInterface()
 experimental/typecons.d:126:private template 
 implementsInterface(Source,
 Targets...)
 experimental/typecons.d:184:template wrap(Targets...)
 experimental/typecons.d:237:        template wrap(Source)
 range/package.d:2069:template Take(R)
 range/package.d:3501:template Cycle(R)
 range/interfaces.d:277:template MostDerivedInputRange(R)
 range/interfaces.d:336:template InputRangeObject(R)
 numeric.d:678:template FPTemporary(F)
 conv.d:3894:template octal(alias decimalInteger)
 utf.d:1136:package template codeUnitLimit(S)
 typecons.d:1779:private mixin template RebindableCommon(T, U, 
 alias This)
 typecons.d:1838:template Rebindable(T)
 typecons.d:4239:template wrap(Targets...)
 typecons.d:4252:    template wrap(Source)
 typecons.d:4412:template wrap(Targets...)
 typecons.d:4429:template unwrap(Target)
 typecons.d:4461:template unwrap(Target)
 format.d:657:template FormatSpec(Char)
 algorithm/iteration.d:1055:template filter(alias predicate) if 
 (is(typeof
 (unaryFun!predicate)))
 internal/math/biguintcore.d:81:template maxBigDigits(T) if 
 (isIntegral!T)
 meta.d:248:package template OldAlias(T) if (!isAggregateType!T 
 || is
 (Unqual!T == T))
 meta.d:254:package template OldAlias(T) if (isAggregateType!T 
 && !is
 (Unqual!T == T))
 utf.d:3542:template byUTF(C) if (isSomeChar!C)


 Appendix B: templates that would need to be extracted out in 
 phobos, if parsing their modules cost a non-negligible amount 
 of time.

 std.meta:
     allSatisfy
     anySatisfy
     ApplyLeft

 std.range.primitives:
     hasSlicing
     isInputRange
     isInfinite

 std.traits:
     isAggregateType
     isAssociativeArray
     isDynamicArray
     isFloatingPoint
     isImplicitlyConvertible
     isIntegral
     isMutable
     isSomeChar


There are a lot of templates in Phobos that never use the 
template keyword. The proposal doesn't only apply to constraints, 
it applies to the whole declaration.
Dec 24 2016
parent reply Chris Wright <dhasenan gmail.com> writes:
On Sat, 24 Dec 2016 09:20:19 +0000, John Colvin wrote:

 There are a lot of templates in Phobos that never use the template
 keyword. The proposal doesn't only apply to constraints, it applies to
 the whole declaration.


If you have a better way of estimating the impact, I'd love to see it.
Dec 24 2016
parent reply Joakim <dlang joakim.fea.st> writes:
On Saturday, 24 December 2016 at 17:53:04 UTC, Chris Wright wrote:

 On Sat, 24 Dec 2016 09:20:19 +0000, John Colvin wrote:

 There are a lot of templates in Phobos that never use the 
 template keyword. The proposal doesn't only apply to 
 constraints, it applies to the whole declaration.


 If you have a better way of estimating the impact, I'd love to 
 see it.


Can we hold off on the performance discussion?  Walter says this 
DIP isn't hard to implement 
(https://github.com/dlang/DIPs/pull/51#issuecomment-269077790), 
so we will run some numbers and see what we get.  As you know, I 
too am skeptical of the benefit but Andrei has shown that import 
fanout has a non-negligible cost with his latest benchmark, and 
actual measurement will be the best way to decide.
Dec 24 2016
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/24/2016 02:16 PM, Joakim wrote:

 On Saturday, 24 December 2016 at 17:53:04 UTC, Chris Wright wrote:

 On Sat, 24 Dec 2016 09:20:19 +0000, John Colvin wrote:

 There are a lot of templates in Phobos that never use the template
 keyword. The proposal doesn't only apply to constraints, it applies
 to the whole declaration.


 If you have a better way of estimating the impact, I'd love to see it.


 Can we hold off on the performance discussion?  Walter says this DIP
 isn't hard to implement
 (https://github.com/dlang/DIPs/pull/51#issuecomment-269077790), so we
 will run some numbers and see what we get.  As you know, I too am
 skeptical of the benefit but Andrei has shown that import fanout has a
 non-negligible cost with his latest benchmark, and actual measurement
 will be the best way to decide.


Also (from the cycle "sounding like a broken record") the impact is in 
other dimension than import speed. From DIP1005:

==========
[...] Dependency-Carrying Declarations have multiple benefits:

* Specifies dependencies at declaration level, not at module level. This 
allows reasoning about the dependency cost of declarations in separation 
instead of aggregated at module level.

* If all declarations use Dependency-Carrying style and there is no 
top-level import, human reviewers and maintainers can immediately tell 
where each symbol in a given declaration comes from. This is a highly 
nontrivial exercise without specialized editor support in projects that 
pull several other modules and packages wholesale. Even a project 
newcomer could gather an understanding of a declaration without needing 
to absorb an arbitrary amount of implied context from the declaration at 
the top of the module.

* Dependency-Carrying Declarations are easier to move around, making for 
simpler and faster refactorings.

* Dependency-Carrying Declarations allow scalable template libraries. [...]

Dependency-Carrying Declarations also have drawbacks:

* If most declarations in a module need the same imports, then factoring 
them outside the declarations at top level is simpler and better than 
repeating them.

* Related, renaming one module is likely to require more edits in a 
Dependency-Carrying Declarations setup.

* Traditional dependency-tracking tools such as make and other build 
systems assume file-level dependencies and need special tooling (such as 
rdmd) in order to work efficiently.

* Dependencies at the top of a module are easier to inspect quickly than 
dependencies spread through the module.
==========


Andrei
Dec 24 2016
prev sibling parent deadalnix <deadalnix gmail.com> writes:
On Saturday, 24 December 2016 at 19:16:40 UTC, Joakim wrote:

 Can we hold off on the performance discussion?  Walter says 
 this DIP isn't hard to implement 
 (https://github.com/dlang/DIPs/pull/51#issuecomment-269077790), 
 so we will run some numbers and see what we get.  As you know, 
 I too am skeptical of the benefit but Andrei has shown that 
 import fanout has a non-negligible cost with his latest 
 benchmark, and actual measurement will be the best way to 
 decide.


The "problem" is that you'd be essentially measuring how 
inefficient the current DMD import management is rather than 
anything else.

When you push something on the user, you want to make sure it 
bring an actual benefit, not just allow to work around lousy 
existing implementation.
Dec 28 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/23/2016 09:23 PM, Chris Wright wrote:
[abbreviated below]

Upon more investigation, I see a large discrepancy between the findings 
of DIP1005 and yours. So there are a number of claims here, which I'll 
summarize:


 tldr: the performance impact of this DIP would be too small to easily
 measure and only impacts 26 declarations referencing 14 templates in the
 standard library anyway.



 Phobos has 26 templates that would use this special syntax, referencing
 14 distinct templates and CTFE functions in three modules.



 The templates that have to be parsed are std.traits, std.meta, and
 std.range.primitives. I wrote a test program that imported them without
 using them. It compiled in too little time to accurately measure --
 `time` reports 0.01s wall time.

 You want to add new language syntax to save me ten milliseconds during
 compilation.


The findings of DIP1005 are the following:

* Importing a single std module also imports on average 10.5 other modules.

* Importing a single std module costs on average 64.6 ms.

* (Not stated in the DIP) A majority of std templates would acquire 
inline imports.

According to the DIP, one may estimate that the proposed feature would 
reduce additional imports to 0 and the average time to import a single 
module by a factor of 10 to under 10 ms.

By your estimates:

* 26 templates in std need inline imports.

* Importing a single std module today would only imports 1-3 other 
modules most of the time (one or more of std.traits, std.meta, and 
std.range.primitives).

* These additional imports cost in aggregate under 10ms, bringing the 
average cost of importing a module itself to 54.6 ms.

* It follows that the average module takes 5.46 more times to import 
alone than the sum of std.traits, std.meta, and std.range.primitives 
(which have a total of 11263 lines, 5x more than the average Phobos module).

I don't see how your claims can be simultaneously true with the findings 
of DIP1005. The scripts that compute those numbers are available with 
the DIP. Were you able to reproduce them?

(I confirm that importing std.traits, std.meta, and std.range.primitives 
together takes 10ms.)


Thanks,

Andrei
Dec 24 2016
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 24 December 2016 at 09:34:03 UTC, Andrei 
Alexandrescu wrote:

 (I confirm that importing std.traits, std.meta, and 
 std.range.primitives together takes 10ms.)


while compiling std.traits 6 files are opened and read into 
memory,
taking roughly 300 microseconds, which is not even 0.3% of the 
time spent.
Lexing them requires additionally also about 300 microseconds.
So Together that makes up 0.6% of the time spent.

The real problem here is _NOT_ opening and lexing files.

It is rather eager parsing and sema.

If that were made more lazy, we could import half of the world 
with noticing impact.

(Which espcially in std.traits, would not make that much of a 
difference since every template in there depends on nearly every 
other template in there)
Dec 24 2016
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 24 December 2016 at 10:54:08 UTC, Stefan Koch wrote:

 300 microseconds, which is not even 0.3% of the time spent.
 Lexing them requires additionally also about 300 microseconds.
 So Together that makes up 0.6% of the time spent.


Of course in the above 3% and 6% are the right numbers.
(And still conservative.)
Since the are obtained by using a -profile build of dmd.
Dec 24 2016
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/24/2016 05:54 AM, Stefan Koch wrote:

 If that were made more lazy, we could import half of the world with
 noticing impact.


That is what 1005 will bring. -- Andrei
Dec 24 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 24 December 2016 at 14:08:48 UTC, Andrei 
Alexandrescu wrote:

 On 12/24/2016 05:54 AM, Stefan Koch wrote:

 If that were made more lazy, we could import half of the world 
 with
 noticing impact.


 That is what 1005 will bring. -- Andrei


A compiler enhancement can do this _without_ a language change.
Dec 24 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/24/16 9:20 AM, Stefan Koch wrote:

 On Saturday, 24 December 2016 at 14:08:48 UTC, Andrei Alexandrescu wrote:

 On 12/24/2016 05:54 AM, Stefan Koch wrote:

 If that were made more lazy, we could import half of the world with
 noticing impact.


 That is what 1005 will bring. -- Andrei


 A compiler enhancement can do this _without_ a language change.


The language addition has additional benefits as described by DIP1005. 
-- Andrei
Dec 24 2016
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 24 December 2016 at 15:44:18 UTC, Andrei 
Alexandrescu wrote:

 On 12/24/16 9:20 AM, Stefan Koch wrote:

 On Saturday, 24 December 2016 at 14:08:48 UTC, Andrei 
 Alexandrescu wrote:

 On 12/24/2016 05:54 AM, Stefan Koch wrote:

 If that were made more lazy, we could import half of the 
 world with
 noticing impact.


 That is what 1005 will bring. -- Andrei


 A compiler enhancement can do this _without_ a language change.


 The language addition has additional benefits as described by 
 DIP1005. -- Andrei


I just read over the dip, and it is a giant wall of text.
I cannot really make heads or tails of it.
Maybe you could write the advantages you hit at in short 
bullet-point form ?
Dec 24 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/24/2016 12:45 PM, Stefan Koch wrote:

 I just read over the dip, and it is a giant wall of text.
 I cannot really make heads or tails of it.


When it was shorter, people complained it had too little information.

If you're using the text form, the nicey formatted version may be 
helpful: 
https://github.com/dlang/DIPs/blob/0e15d550d6774dfd40ec78dc201dfad33f47a740/DIPs/DIP1005.md


 Maybe you could write the advantages you hit at in short bullet-point
 form ?


There is. Search for "The analysis above reveals that 
Dependency-Carrying Declarations have multiple benefits:".


Thanks,

Andrei
Dec 24 2016
parent Chris Wright <dhasenan gmail.com> writes:
On Sat, 24 Dec 2016 14:10:30 -0500, Andrei Alexandrescu wrote:


 On 12/24/2016 12:45 PM, Stefan Koch wrote:

 I just read over the dip, and it is a giant wall of text.
 I cannot really make heads or tails of it.


 
 When it was shorter, people complained it had too little information.


It's not the length that's giving trouble; it's the organization.
Dec 24 2016
prev sibling parent Chris Wright <dhasenan gmail.com> writes:
On Sat, 24 Dec 2016 17:45:35 +0000, Stefan Koch wrote:

 I just read over the dip, and it is a giant wall of text.
 I cannot really make heads or tails of it.
 Maybe you could write the advantages you hit at in short bullet-point
 form ?


We add the syntax:

with (import foo, bar, baz)
SingleDeclaration

* If you're reading it as source code, you have, usually, 1-3 modules to 
look in for related definitions, instead of lots (plain imports) or one 
(static / selective imports).

* These modules are listed one line up from the declaration (as with 
static imports, unlike selective imports).

* It doesn't look quite as cluttered as static imports.

* It saves a little typing compared to static imports when you use the 
same import twice in one declaration.

* If you need to move this declaration to another module, you don't have 
to worry about imports (provided you assiduously avoid global imports). 
You still have to worry about everything the declaration references from 
the module it was previously defined in.

* The compiler can delay importing the module until you use the 
declaration it's attached to.

* It might be slightly easier to implement in DMD than lazy static / 
selective imports.



We extend that syntax:

with (import foo)
with (import bar, baz)
{
  // multiple declarations
  // possibly other with-import blocks
}

* You have even less typing if you reuse the same import in a lot of 
declarations.

* The compiler can still delay importing the module until you use the 
declaration it's attached to.

* You get to have a fun search through the entire module to try to find 
which modules are imported when reading a declaration, instead of looking 
up one line (first pass on the syntax), looking at the symbol being used 
(static imports), or looking at the top of the file (status quo or 
selective imports).

* That also applies when you try to move a declaration to another module.
Dec 24 2016
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Saturday, 24 December 2016 at 15:44:18 UTC, Andrei 
Alexandrescu wrote:

 A compiler enhancement can do this _without_ a language change.


 The language addition has additional benefits as described by 
 DIP1005. -- Andrei


Yes, question is, are these specific benefits worth adding a 
language change ?

Right now you got :

A/ No language change, get X.
B/ Language change, get X and Y.

It stand to reason that B should be evaluated on the benefit 
provided by Y, and Y only, rather than X and Y, as X can be 
provided without the language change.
Dec 28 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/28/16 10:48 AM, deadalnix wrote:

 On Saturday, 24 December 2016 at 15:44:18 UTC, Andrei Alexandrescu wrote:

 A compiler enhancement can do this _without_ a language change.


 The language addition has additional benefits as described by DIP1005.
 -- Andrei


 Yes, question is, are these specific benefits worth adding a language
 change ?

 Right now you got :

 A/ No language change, get X.
 B/ Language change, get X and Y.

 It stand to reason that B should be evaluated on the benefit provided by
 Y, and Y only, rather than X and Y, as X can be provided without the
 language change.


This is exactly what the DIP describes in detail. It consecrates 
sections to alternatives. Is anything missing? -- Andrei
Dec 28 2016
parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 28 December 2016 at 23:14:48 UTC, Andrei 
Alexandrescu wrote:

 On 12/28/16 10:48 AM, deadalnix wrote:

 On Saturday, 24 December 2016 at 15:44:18 UTC, Andrei 
 Alexandrescu wrote:

 A compiler enhancement can do this _without_ a language 
 change.


 The language addition has additional benefits as described by 
 DIP1005.
 -- Andrei


 Yes, question is, are these specific benefits worth adding a 
 language
 change ?

 Right now you got :

 A/ No language change, get X.
 B/ Language change, get X and Y.

 It stand to reason that B should be evaluated on the benefit 
 provided by
 Y, and Y only, rather than X and Y, as X can be provided 
 without the
 language change.


 This is exactly what the DIP describes in detail. It 
 consecrates sections to alternatives. Is anything missing? -- 
 Andrei


This was more a comment about the current discussion than the 
DIP. A lot of discussion effort was focused on performance, but 
most of it can be achieved without language change - as the DIP 
states.

I think the performance gain we are looking at here is marginal, 
and I don't expect people to change their code to get a marginal 
benefit, so I suggest the performance aspect of the change to be 
simply left aside.

So the question now is would the added expressivity of per 
declaration import be worth the language change. I have to admit 
I'm not convinced either way.

Finally, while I proposed a variation of the "with import" combo 
in the past, I'm now much more convinced that using the plain 
import syntax, without the ';' is better. I was afraid there was 
a syntax conflict, but it doesn't looks like there is one. This 
will not introduce a new syntax, but allow an existing one to be 
used in a new location. This is, IMO, much more valuable.
Dec 30 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 30 December 2016 at 18:11:54 UTC, deadalnix wrote:


 I think the performance gain we are looking at here is 
 marginal, and I don't expect people to change their code to get 
 a marginal benefit, so I suggest the performance aspect of the 
 change to be simply left aside.


Yes, imports are (in most cases) not performance relevant!


 So the question now is would the added expressivity of per 
 declaration import be worth the language change. I have to 
 admit I'm not convinced either way.


Neither am I.


 Finally, while I proposed a variation of the "with import" 
 combo in the past, I'm now much more convinced that using the 
 plain import syntax, without the ';' is better. I was afraid 
 there was a syntax conflict, but it doesn't looks like there is 
 one. This will not introduce a new syntax, but allow an 
 existing one to be used in a new location. This is, IMO, much 
 more valuable.


I am still not sure what problem it is trying to solve :)
Dec 30 2016
parent reply Chris Wright <dhasenan gmail.com> writes:
On Fri, 30 Dec 2016 18:18:12 +0000, Stefan Koch wrote:

 I am still not sure what problem it is trying to solve :)


* Performance improvements, primarily when a module imports another, 
bulky module for optional functionality.
* Making it easier to locate where things are defined when reading code.
* Making it easier to move declarations between files.
Dec 30 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 30 December 2016 at 22:29:18 UTC, Chris Wright wrote:

 * Performance improvements, primarily when a module imports 
 another,
 bulky module for optional functionality.


That is solved by selective imports.


 * Making it easier to locate where things are defined when 
 reading code.


That is solved by selective imports.


 * Making it easier to move declarations between files.


That is solved by selective imports.

Inline imports are really just a addition retrive the the 
deprecated behavior of fullyQualifedNames which would implicitly 
import.
Dec 30 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Stefan Koch <uplink.coder googlemail.com> wrote:

 On Friday, 30 December 2016 at 22:29:18 UTC, Chris Wright wrote:

 * Performance improvements, primarily when a module imports 
 another,
 bulky module for optional functionality.


 That is solved by selective imports.
 

 * Making it easier to locate where things are defined when 
 reading code.


 
 That is solved by selective imports.
 

 * Making it easier to move declarations between files.


 
 That is solved by selective imports.
 
 Inline imports are really just a addition retrive the the 
 deprecated behavior of fullyQualifedNames which would implicitly 
 import.
 


The main win, which indeed is not emphasized enough, is better
encapsulation. Walter pointed that out, and I will redo the DIP to put that
front and center.
Dec 30 2016
next sibling parent ArturG <var.spool.mail700 gmail.com> writes:
On Friday, 30 December 2016 at 23:49:23 UTC, Andrei Alexandrescu 
wrote:


 The main win, which indeed is not emphasized enough, is better 
 encapsulation. Walter pointed that out, and I will redo the DIP 
 to put that front and center.


Maybe i can provide an example where i think DCD's would be 
usefull.
Im experimenting with a more Qt like signals and slots 
implementation,
it introspects some type for function declarations with the 
 ("Signal") uda
and mixes them into the current type.
It currently looks like this:

// moda.d
interface TestSignals
{
    import modb: TestType;
     ("Signal"):
       void someTestSig(TestType);
}

// modb.d
struct TestType{}

// test.d
class A : SignalObject
{
    import moda: TestSignals;
    import modb: TestType; // has to be manually imported for the 
generated signal

    mixin signalsOf!(SignalList, TestSignals);

    interface SignalList
    {
        ("Signal"):
          void foo(int);
          void bar(string, int = 100);
          int bar(int a, int b);
    }

    void someFun(string s) { s.writeln; }
}

class B : SignalObject
{
    struct SignalList
    {
        ("Signal"):
          void someSig(string);
    }

    mixin signalsOf!SignalList;

    void fooHandler(int i){ i.writeln; }

    int onBar(int a, int b){ writeln(a + b); return a + b; }
    void onBar(string s, int i){ writeln(s, i); }

    import modb: TestType;
    void someTestSigHandler(TestType t){ t.writeln; }
}

class C : SignalObject
{
    import modb: TestType; // has to be manually imported for the 
generated signal

    mixin signalsOf!(A, B);

    // i wanted to support string based signal declaration
    // but that would require double manual import declarations

    // import modb: TestType;

    // enum signalList =
    // q{
    //    import modb: TestType;
    //     ("Signal"):
    //    void someSig(TestType);
    //    int someOtherSig();
    // }

    // mixin signalsOf!signalList;
}

void main(string[] args)
{
    auto a = new A;
    auto b = new B;
    auto c = new C;

    a.connect!"foo"(&b.fooHandler);
    a.connect(b); // connect all bar's with all onBar's

    a.foo(66);
    a.bar("asd");
    a.bar(4, 6);

    c.connect!"someSig"(&b.someSig);
    b.connect!"someSig"(&a.someFun);

    c.someSig("emitted by c forwarded by b handled by a");

    a.connect!"someTestSig"(&b.someTestSigHandler);
    c.connect!"someTestSig"(&b.someTestSigHandler);

    import modb: TestType;
    TestType t;
    a.someTestSig(t);
    c.someTestSig(t);
}

i assume with DCD's i could remove those manual imports if they 
get some traits.
Dec 30 2016
prev sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Fri, 30 Dec 2016 23:49:23 +0000, Andrei Alexandrescu wrote:

 The main win, which indeed is not emphasized enough, is better
 encapsulation. Walter pointed that out, and I will redo the DIP to put
 that front and center.


Encapsulation is an abstraction over several concrete benefits. We're not 
horribly constrained in time or cognitive effort, so we can talk about 
the concrete benefits instead of the abstraction.
Dec 30 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/30/16 7:34 PM, Chris Wright wrote:

 On Fri, 30 Dec 2016 23:49:23 +0000, Andrei Alexandrescu wrote:

 The main win, which indeed is not emphasized enough, is better
 encapsulation. Walter pointed that out, and I will redo the DIP to put
 that front and center.


 Encapsulation is an abstraction over several concrete benefits. We're not
 horribly constrained in time or cognitive effort, so we can talk about
 the concrete benefits instead of the abstraction.


Framing things this way would be really ignorant. DIP1005 can't be in 
the business of arguing that encapsulation is good by means of examples. 
First off, there is no undeniable proof that encapsulation is 
advantageous and not even simple obvious examples; encapsulation has had 
many opponents, most notably Fred Brooks (the author of "The Mythical 
Man-Month"), who used a process specifically antithetic to encapsulation 
in the development of IBM System/360 and OS/360. Far as I recall Fred 
ultimately ceded the point that encapsulation is superior, but IIRC that 
was after 2000. I have no doubt there are competent folks out there who 
think encapsulation is a crock.

So it would be goofy if DIP1005 took the onus to show the advantages of 
improved encapsulation by means of concrete examples (such don't exist 
outside of large-scale projects for any kind of encapsulation). What 
DIP1005 can do is show to someone who already believes that 
encapsulation is good, that the proposed feature improves encapsulation.


Andrei
Dec 30 2016
parent reply Chris Wright <dhasenan gmail.com> writes:
On Fri, 30 Dec 2016 21:13:19 -0500, Andrei Alexandrescu wrote:

 DIP1005 can't be in
 the business of arguing that encapsulation is good by means of examples.


Right. I said we should talk about the concrete benefits of the proposal 
instead of talking about encapsulation.
Dec 30 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/30/16 11:10 PM, Chris Wright wrote:

 On Fri, 30 Dec 2016 21:13:19 -0500, Andrei Alexandrescu wrote:

 DIP1005 can't be in
 the business of arguing that encapsulation is good by means of examples.


 Right. I said we should talk about the concrete benefits of the proposal
 instead of talking about encapsulation.


Why would the DIP hamstring itself by not discussing its most important 
advantage? -- Andrei
Dec 31 2016
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 31.12.2016 13:23, Andrei Alexandrescu wrote:

 On 12/30/16 11:10 PM, Chris Wright wrote:

 On Fri, 30 Dec 2016 21:13:19 -0500, Andrei Alexandrescu wrote:

 DIP1005 can't be in
 the business of arguing that encapsulation is good by means of examples.


 Right. I said we should talk about the concrete benefits of the proposal
 instead of talking about encapsulation.


 Why would the DIP hamstring itself by not discussing its most important
 advantage? -- Andrei


Because it is a slippery slope if you subscribe to the notion that "D's 
unit of encapsulation is the module".

Do we need to rethink encapsulation in D? Should there be an additional 
'internal' visibility modifier that hides members even to other 
declarations in the same module?


On an unrelated note: I'm still not a fan of the with(import) syntax as 
it morally promotes a lack of turtles (even if not technically so).
Dec 31 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/31/16 9:43 AM, Timon Gehr wrote:

 On an unrelated note: I'm still not a fan of the with(import) syntax as
 it morally promotes a lack of turtles (even if not technically so).


Could you please provide more detail? Thanks! -- Andrei
Jan 04 2017
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 05.01.2017 01:51, Andrei Alexandrescu wrote:

 On 12/31/16 9:43 AM, Timon Gehr wrote:

 On an unrelated note: I'm still not a fan of the with(import) syntax as
 it morally promotes a lack of turtles (even if not technically so).


 Could you please provide more detail? Thanks! -- Andrei


The declaration

with(import foo){ ... }

looks like an orthogonal combination of some import expression and the usual

with(foo){ ... }

statement. (This is true for all other statements and expressions with 
similar syntax.)

However, this is not in fact true here, the two constructs have 
different scoping rules.

Hence I think that the 'with(import foo){ ... }'-syntax would be better 
split into two orthogonal features:

1. allow 'import foo' as an expression that evaluates to the 
corresponding module symbol.

2. add 'static with' that is basically like 'with' but is a declaration 
and has different scoping rules.


I.e., my objection is that 'with' should not become 'static' just 
because it is applied to an import expression.
Jan 04 2017
prev sibling parent Chris Wright <dhasenan gmail.com> writes:
On Sat, 31 Dec 2016 07:23:14 -0500, Andrei Alexandrescu wrote:


 On 12/30/16 11:10 PM, Chris Wright wrote:

 On Fri, 30 Dec 2016 21:13:19 -0500, Andrei Alexandrescu wrote:

 DIP1005 can't be in the business of arguing that encapsulation is good
 by means of examples.


 Right. I said we should talk about the concrete benefits of the
 proposal instead of talking about encapsulation.


 
 Why would the DIP hamstring itself by not discussing its most important
 advantage? -- Andrei


DIP1005 provides concrete advantages. We can productively discuss its 
concrete advantages.

"Encapsulation" is an abstraction over a number of features. Many 
features can increase potential encapsulation, and this is only one of 
them. Encapsulation can provide a wide range of benefits, and this 
feature only provides one or two of them.

It saves us no effort to say "encapsulation". We still have to peel back 
that label to see what's underneath and evaluate that. But 
"encapsulation" is an applause light, so we automatically think better of 
the proposal than if we were merely looking at its concrete benefits.
Dec 31 2016
prev sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Fri, 30 Dec 2016 22:42:39 +0000, Stefan Koch wrote:


 On Friday, 30 December 2016 at 22:29:18 UTC, Chris Wright wrote:

 * Performance improvements, primarily when a module imports another,
 bulky module for optional functionality.


 That is solved by selective imports.


Well, not today, but it could be. Same with static imports. Both 
accomplish this goal better than DIP1005.


 * Making it easier to locate where things are defined when reading
 code.


 
 That is solved by selective imports.


Static imports do that better than selective imports, though it's more 
typing.

With selective imports, you can either find the declaration in the 
current module, or find the symbol in an import list. If it's imported, 
and the module author is using standard code formatting, you will find 
the import at the top of the module, which will be fast. This is less 
true with arbitrarily scoped imports.

With DIP1005, you rely on there also being selective or static imports. 
If the module author didn't use them, then you have to pull out grep.


 * Making it easier to move declarations between files.


 
 That is solved by selective imports.


That helps a bit. You still need to exercise discipline in only using 
selective imports for what you need for visible declarations.

Static imports work about as well, except you can probably write a 
complicated regex replacement to extract out the imports you need.

DIP1005 is a solid improvement here.

Unfortunately, this is a rather marginal usecase.

-- Another thing that just occurred to me: if you're modifying a function 
signature and that brings in another imported symbol, you don't have to 
move to the top of the file or struct to add the necessary import. It's a 
small thing for vim users, who are used to using marks and fast movement 
commands, but if you're using, say, VSCode, that might be painful.
Dec 30 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/30/16 7:32 PM, Chris Wright wrote:

 On Fri, 30 Dec 2016 22:42:39 +0000, Stefan Koch wrote:


 On Friday, 30 December 2016 at 22:29:18 UTC, Chris Wright wrote:

 * Performance improvements, primarily when a module imports another,
 bulky module for optional functionality.


 That is solved by selective imports.


 Well, not today, but it could be. Same with static imports. Both
 accomplish this goal better than DIP1005.


Is this fact or opinion? If the former, could you please point where 
DIP1005 is getting it wrong. Thanks.


 * Making it easier to locate where things are defined when reading
 code.


 That is solved by selective imports.


 Static imports do that better than selective imports, though it's more
 typing.


So whether that's overall better is not settled, is it? We can't really 
define "better" as whatever each participant believes.


 With selective imports, you can either find the declaration in the
 current module, or find the symbol in an import list. If it's imported,
 and the module author is using standard code formatting, you will find
 the import at the top of the module, which will be fast. This is less
 true with arbitrarily scoped imports.

 With DIP1005, you rely on there also being selective or static imports.
 If the module author didn't use them, then you have to pull out grep.


DIP1005 allows the declaration to encapsulate its own dependencies. Of 
course if top-level imports are also present, the benefit erodes.


 * Making it easier to move declarations between files.


 That is solved by selective imports.


 That helps a bit. You still need to exercise discipline in only using
 selective imports for what you need for visible declarations.

 Static imports work about as well, except you can probably write a
 complicated regex replacement to extract out the imports you need.

 DIP1005 is a solid improvement here.

 Unfortunately, this is a rather marginal usecase.

 -- Another thing that just occurred to me: if you're modifying a function
 signature and that brings in another imported symbol, you don't have to
 move to the top of the file or struct to add the necessary import. It's a
 small thing for vim users, who are used to using marks and fast movement
 commands, but if you're using, say, VSCode, that might be painful.


That's one of the many benefits of encapsulation.


Andrei
Dec 30 2016
parent reply Chris Wright <dhasenan gmail.com> writes:
On Fri, 30 Dec 2016 20:56:54 -0500, Andrei Alexandrescu wrote:


 On 12/30/16 7:32 PM, Chris Wright wrote:

 On Fri, 30 Dec 2016 22:42:39 +0000, Stefan Koch wrote:


 On Friday, 30 December 2016 at 22:29:18 UTC, Chris Wright wrote:

 * Performance improvements, primarily when a module imports another,
 bulky module for optional functionality.


 That is solved by selective imports.


 Well, not today, but it could be. Same with static imports. Both
 accomplish this goal better than DIP1005.


 
 Is this fact or opinion? If the former, could you please point where
 DIP1005 is getting it wrong. Thanks.


Consider:

  with(import std.range.primitives, std.traits)
  {
    template Foo(Range) if (isForwardRange!Range && !isArray!Range) {}
    template Bar(Range) if (isArray!Range) {}
  }

If I invoke Bar, the compiler must read and parse both 
std.range.primitives and std.traits.

With static imports:

  static import std.range.primitives, std.traits;
  template Foo(Range) if (std.range.primitives.isForwardRange!Range
      && !std.traits.isArray!Range) {}
  template Bar(Range) if (std.traits.isArray!Range) {}

If I invoke Bar, the compiler must read and parse std.traits but not 
std.range.primitives because it is unambiguous that isArray is in 
std.traits, not std.range.primivites.

With selective imports likewise.

You're going to respond that I'm only complaining that people can abuse 
the syntax. Consider that it also happens if I change the code and forget 
to update the imports. Also consider that people are likely to 
incorporate several related definitions that tend to use the same imports 
in order to save typing. "Be more virtuous" isn't a mantra that actually 
leads to better code.


 * Making it easier to locate where things are defined when reading
 code.


 That is solved by selective imports.


 Static imports do that better than selective imports, though it's more
 typing.


 
 So whether that's overall better is not settled, is it? We can't really
 define "better" as whatever each participant believes.


Whether the feature is overall better is not settled. Whether locating 
where symbols are defined is easier with DIP1005 than with static imports 
*is*.

Static imports tell you exactly where the symbol is defined at a glance, 
whereas with DIP1005, you have to look through at least two modules.


 With selective imports, you can either find the declaration in the
 current module, or find the symbol in an import list. If it's imported,
 and the module author is using standard code formatting, you will find
 the import at the top of the module, which will be fast. This is less
 true with arbitrarily scoped imports.

 With DIP1005, you rely on there also being selective or static imports.
 If the module author didn't use them, then you have to pull out grep.


 
 DIP1005 allows the declaration to encapsulate its own dependencies. Of
 course if top-level imports are also present, the benefit erodes.


Consider:

  // tons of code
  with (import std.traits)
  template Bar(Range) if (isArray!Range && !isAbstractClass!Range) {}
  // tons more code

I can reasonably guess that the import is required for at least one of 
isArray and isAbstractClass, though for all I know, if I'm unfamiliar 
with std.traits, the import is unnecessary and both isArray and 
isAbstractClass are defined in the current module.

You can fix that with a static or selective import:

  with (import std.traits : isArray, isAbstractClass)
  template Bar(Range) if (isArray!Range && !isAbstractClass!Range) {}

Far less ambiguous, but your examples all have unmodified with-imports, 
and you seem less than keen about selective imports.


 -- Another thing that just occurred to me: if you're modifying a
 function signature and that brings in another imported symbol, you
 don't have to move to the top of the file or struct to add the
 necessary import. It's a small thing for vim users, who are used to
 using marks and fast movement commands, but if you're using, say,
 VSCode, that might be painful.


 
 That's one of the many benefits of encapsulation.


It is a benefit of this DIP. There are many ways to achieve 
encapsulation, and there are many effects of encapsulation, and most of 
that is irrelevant to this discussion.

FWIW, you could also use current imports immediately before the relevant 
declaration. That would be a bad idea in general because it makes it much 
harder to locate imports when trying to locate a definition.
Dec 31 2016
parent Chris Wright <dhasenan gmail.com> writes:
On Sat, 31 Dec 2016 16:41:16 +0000, Chris Wright wrote:

 Far less ambiguous, but your examples all have unmodified with-imports,
 and you seem less than keen about selective imports.


Correction: *tend to* have unmodified with-imports.
Dec 31 2016
prev sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Wed, 28 Dec 2016 15:48:46 +0000, deadalnix wrote:


 On Saturday, 24 December 2016 at 15:44:18 UTC, Andrei Alexandrescu
 wrote:

 A compiler enhancement can do this _without_ a language change.


 The language addition has additional benefits as described by DIP1005.
 -- Andrei


 
 Yes, question is, are these specific benefits worth adding a language
 change ?


And the associated detriments. Just imagine opening up phobos and seeing:

  // 1200 lines of code
  with (import foo)
  {
    // 300 lines of declarations...
    with (import bar)
    {
      // 250 lines of additional declarations...
    }
  }

All mixed in with structs and functions and conditional compilation, so 
you have a sea of curly braces to wade through and indentation is only a 
mild help.
Dec 28 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/28/16 9:47 PM, Chris Wright wrote:

 On Wed, 28 Dec 2016 15:48:46 +0000, deadalnix wrote:


 On Saturday, 24 December 2016 at 15:44:18 UTC, Andrei Alexandrescu
 wrote:

 A compiler enhancement can do this _without_ a language change.


 The language addition has additional benefits as described by DIP1005.
 -- Andrei


 Yes, question is, are these specific benefits worth adding a language
 change ?


 And the associated detriments. Just imagine opening up phobos and seeing:

   // 1200 lines of code
   with (import foo)
   {
     // 300 lines of declarations...
     with (import bar)
     {
       // 250 lines of additional declarations...
     }
   }

 All mixed in with structs and functions and conditional compilation, so
 you have a sea of curly braces to wade through and indentation is only a
 mild help.


The D Standard Library will generally specify dependencies with 
declarations. If the point was that the feature can be misused or abused 
if one really tries, I agree. -- Andrei
Dec 29 2016
prev sibling parent reply Martin Nowak <code dawg.eu> writes:
On Saturday, 24 December 2016 at 10:54:08 UTC, Stefan Koch wrote:

 If that were made more lazy, we could import half of the world 
 with noticing impact.

 (Which espcially in std.traits, would not make that much of a 
 difference since every template in there depends on nearly 
 every other template in there)


Also the established technique of serializing precompiled AST 
(after semantic3) of modules to a cache should be applicable as 
well.

Cross-posting from 
https://github.com/dlang/DIPs/pull/51#issuecomment-269107966, b/c 
it wasn't answered yet.

Were any other means considered? This is proposing to add plenty 
of additional annotations only to speed up compilation, but none 
of the classical tools for pre-compilation were assessed.
Since D's modules don't have the header problem, even 
pre-compilation and reuse of semantic3 should be possible, or not?
Dec 31 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/31/2016 12:20 PM, Martin Nowak wrote:

 On Saturday, 24 December 2016 at 10:54:08 UTC, Stefan Koch wrote:

 If that were made more lazy, we could import half of the world with
 noticing impact.

 (Which espcially in std.traits, would not make that much of a
 difference since every template in there depends on nearly every other
 template in there)


 Also the established technique of serializing precompiled AST (after
 semantic3) of modules to a cache should be applicable as well.

 Cross-posting from
 https://github.com/dlang/DIPs/pull/51#issuecomment-269107966, b/c it
 wasn't answered yet.


No worries, I'll make a pass (a rewrite, really) taking all feedback 
into account.


 Were any other means considered? This is proposing to add plenty of
 additional annotations only to speed up compilation, but none of the
 classical tools for pre-compilation were assessed.
 Since D's modules don't have the header problem, even pre-compilation
 and reuse of semantic3 should be possible, or not?


DIP1005 gives consideration to the speed of compilation aspect in larger 
proportion than speed's importance; the first and foremost benefit of 
DIP1005 is it closes the gap on dependency encapsulation, which had been 
very successfully narrowed by local imports. However, the DIP will keep 
the experiments and results on speed measurements because they are 
relevant to it and any related methods of lazily loading of modules.

Regarding the ongoing doubts about the advantages of inline imports: 
they are first and foremost a completion of the nested import feature. 
As such, most, if not all, arguments against inline imports apply 
equally to nested imports. Come to think of it, lazy imports vs nested 
imports:

* same improvement in compilation speed? check
* no language changes? check
* no nasty bugs in the aftermath (such as the infamous 
https://issues.dlang.org/show_bug.cgi?id=10378)? check
* scalable builds? check

Yet local imports are overwhelmingly superior to lazy imports because of 
one thing: they localize dependencies. They introduce modularity and its 
ancillary perks (fast and scalable builds, easier review and 
refactoring) not by engineering, but by organically placing dependencies 
spatially with their dependents. (The scope statement does the same 
thing with temporal dependencies.) That the DIP does not make it clear 
that it is a necessary and sufficient extension of local imports is a 
problem with it.

I now am really glad we slipped local imports before the formalization 
of DIPs. The feature could have been easily demeaned out of existence.

Allow me to make an appeal regarding the review of any DIP. There seems 
to be a tendency of some reviewers to get attached and emotionally 
invested to their opinion, to the extent they'd be hurt by being "wrong" 
and would go to great lengths to argue they're "right". This has obvious 
negative effects on the entire process. Please don't. There's no loss of 
face to worry about. The only commitment we all should have is to the 
good of the D language. If DIP1005 reaches the conclusion of its own 
uselessness, I'd be the first one to write it up and close the PR.


Thanks,

Andrei
Jan 02 2017
next sibling parent reply Dominikus Dittes Scherkl <Dominikus.Scherkl continental-corporation.com> writes:
On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu 
wrote:

 DIP1005 gives consideration to the speed of compilation aspect 
 in larger proportion than speed's importance; the first and 
 foremost benefit of DIP1005 is it closes the gap on dependency 
 encapsulation, which had been very successfully narrowed by 
 local imports.


I love that idea. But I still can't see why this requires any new 
syntax. Simply extending the scope of local inports to include 
the function header is enough.
Only for .di-generation it may be useful to move all local 
imports to the declaration (maybe with this new syntax "with" 
before it) - but that should be done with ALL local imports, 
because today the .di-files are incomplete and will stay so if 
the new syntax is introduced but "old-style" local imports still 
valid and not exported to the .di.
Or the old local imports become deprecated together with the 
introduction of the new "with" syntax and vanish soon after that.
Jan 02 2017
next sibling parent Chris Wright <dhasenan gmail.com> writes:
On Tue, 03 Jan 2017 07:19:41 +0000, Dominikus Dittes Scherkl wrote:


 On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu wrote:

 DIP1005 gives consideration to the speed of compilation aspect in
 larger proportion than speed's importance; the first and foremost
 benefit of DIP1005 is it closes the gap on dependency encapsulation,
 which had been very successfully narrowed by local imports.


 I love that idea. But I still can't see why this requires any new
 syntax. Simply extending the scope of local inports to include the
 function header is enough.


You mean:

  Socket connect()
  {
    import std.socket;
    return new Socket();
  }

Which should already work if you used auto for the return type.
Jan 03 2017
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 01/03/2017 02:19 AM, Dominikus Dittes Scherkl wrote:

 On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu wrote:

 DIP1005 gives consideration to the speed of compilation aspect in
 larger proportion than speed's importance; the first and foremost
 benefit of DIP1005 is it closes the gap on dependency encapsulation,
 which had been very successfully narrowed by local imports.


 I love that idea. But I still can't see why this requires any new
 syntax. Simply extending the scope of local inports to include the
 function header is enough.


I'll mention this possibility in the DIP.


 Only for .di-generation it may be useful to move all local imports to
 the declaration (maybe with this new syntax "with" before it) - but that
 should be done with ALL local imports, because today the .di-files are
 incomplete and will stay so if the new syntax is introduced but
 "old-style" local imports still valid and not exported to the .di.
 Or the old local imports become deprecated together with the
 introduction of the new "with" syntax and vanish soon after that.


Local imports don't pertain to the interface, they are an implementation 
detail. Why would those be made a part of the .di?


Andrei
Jan 03 2017
parent Dominikus Dittes Scherkl <Dominikus.Scherkl continental-corporation.com> writes:
On Tuesday, 3 January 2017 at 12:57:22 UTC, Andrei Alexandrescu 
wrote:

 On 01/03/2017 02:19 AM, Dominikus Dittes Scherkl wrote:

 On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu 
 wrote:

 DIP1005 gives consideration to the speed of compilation 
 aspect in
 larger proportion than speed's importance; the first and 
 foremost
 benefit of DIP1005 is it closes the gap on dependency 
 encapsulation,
 which had been very successfully narrowed by local imports.


 I love that idea. But I still can't see why this requires any 
 new
 syntax. Simply extending the scope of local inports to include 
 the
 function header is enough.


 I'll mention this possibility in the DIP.


Good.

 Only for .di-generation it may be useful to move all local 
 imports to
 the declaration (maybe with this new syntax "with" before it) 
 - but that
 should be done with ALL local imports, because today the 
 .di-files are
 incomplete and will stay so if the new syntax is introduced but
 "old-style" local imports still valid and not exported to the 
 .di.
 Or the old local imports become deprecated together with the
 introduction of the new "with" syntax and vanish soon after 
 that.


 Local imports don't pertain to the interface, they are an 
 implementation detail. Why would those be made a part of the 
 .di?


.di are needed to ship with a library. If a function locally 
imports some type, the library is dependant on that import, so if 
I want to use the library, I have to install all stuff it depends 
on too. And to find out, what exactly the library depends on the 
.di is the place I look, so there need to be mentioned each and 
every import - no matter how deeply local that may be. If that is 
not the case it renders .di-files completely useless to me (as 
they are at the moment), because I need to find out the 
dependencies somewhere else (e.g. in some documentation of the 
library).
While it is likely that the dependencies of a library may be 
documented somewhere, this is not guaranteed. But .di-files are 
guaranteed to ship with a lib because else it cannot link - at 
least if it contains any templates, which is about 100% sure for 
a language like D. So I would like to see local imports in the 
.di-file, even if they are not strictly needed to compile because 
the imported types are not exposed in the function signature.
Jan 04 2017
prev sibling next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu 
wrote:

 Regarding the ongoing doubts about the advantages of inline 
 imports: they are first and foremost a completion of the nested 
 import feature. As such, most, if not all, arguments against 
 inline imports apply equally to nested imports. Come to think 
 of it, lazy imports vs nested imports:

 * same improvement in compilation speed? check
 * no language changes? check
 * no nasty bugs in the aftermath (such as the infamous 
 https://issues.dlang.org/show_bug.cgi?id=10378)? check
 * scalable builds? check

 Yet local imports are overwhelmingly superior to lazy imports 
 because of one thing: they localize dependencies. They 
 introduce modularity and its ancillary perks (fast and scalable 
 builds, easier review and refactoring) not by engineering, but 
 by organically placing dependencies spatially with their 
 dependents. (The scope statement does the same thing with 
 temporal dependencies.) That the DIP does not make it clear 
 that it is a necessary and sufficient extension of local 
 imports is a problem with it.

 I now am really glad we slipped local imports before the 
 formalization of DIPs. The feature could have been easily 
 demeaned out of existence.


Except that almost nobody has argued against local imports.  
Rather, the argument is that local imports mostly solved this 
problem, so why bother adding new syntax for the dozen remaining 
symbols from 2-3 modules that are commonly used in template 
constraints?

Arguing that local imports have been successful so we should 
simply do more of it is not a good argument, as there comes a 
point of diminishing returns.  You need to show that there are 
still worthile gains to be made from changing the language again, 
which is why I want to benchmark this feature with Walter before 
deciding.


 Allow me to make an appeal regarding the review of any DIP. 
 There seems to be a tendency of some reviewers to get attached 
 and emotionally invested to their opinion, to the extent they'd 
 be hurt by being "wrong" and would go to great lengths to argue 
 they're "right". This has obvious negative effects on the 
 entire process. Please don't. There's no loss of face to worry 
 about. The only commitment we all should have is to the good of 
 the D language. If DIP1005 reaches the conclusion of its own 
 uselessness, I'd be the first one to write it up and close the 
 PR.


We could level this analysis back at you: you consider this DIP 
so "obvious" that you are not engaging with our concerns, making 
flip, incorrect remarks about how we would have bikeshedded local 
imports also.

In the end, this is a minor DIP that is easily bikeshedded, as 
everybody can grasp it and have an opinion on it.  I have 
refrained from commenting recently because I will let 
benchmarking settle it for me.  Obviously, that won't suffice for 
others.
Jan 03 2017
next sibling parent Chris Wright <dhasenan gmail.com> writes:
On Tue, 03 Jan 2017 08:10:04 +0000, Joakim wrote:

 Except that almost nobody has argued against local imports. Rather, the
 argument is that local imports mostly solved this problem, so why bother
 adding new syntax for the dozen remaining symbols from 2-3 modules that
 are commonly used in template constraints?


That was my misunderstanding, and it only applied to the speed argument 
anyway.

This is usable with top-level functions, top-level templates, and classes 
with inheritance.
Jan 03 2017
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 01/03/2017 03:10 AM, Joakim wrote:

 On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu wrote:

 Regarding the ongoing doubts about the advantages of inline imports:
 they are first and foremost a completion of the nested import feature.
 As such, most, if not all, arguments against inline imports apply
 equally to nested imports. Come to think of it, lazy imports vs nested
 imports:

 * same improvement in compilation speed? check
 * no language changes? check
 * no nasty bugs in the aftermath (such as the infamous
 https://issues.dlang.org/show_bug.cgi?id=10378)? check
 * scalable builds? check

 Yet local imports are overwhelmingly superior to lazy imports because
 of one thing: they localize dependencies. They introduce modularity
 and its ancillary perks (fast and scalable builds, easier review and
 refactoring) not by engineering, but by organically placing
 dependencies spatially with their dependents. (The scope statement
 does the same thing with temporal dependencies.) That the DIP does not
 make it clear that it is a necessary and sufficient extension of local
 imports is a problem with it.

 I now am really glad we slipped local imports before the formalization
 of DIPs. The feature could have been easily demeaned out of existence.


 Except that almost nobody has argued against local imports.


I don't have time to research this, but my recollection is that at least 
some framed the bugs regarding lookups as a fundamental problem of local 
imports, not a simple matter of getting it right.

Overall, yes, local imports have been a success (really saving 
scalability of large project builds which looked pretty bleak at a 
time), which should increase trust in the authors of the feature... 
hmmm... :o)


 Rather, the
 argument is that local imports mostly solved this problem, so why bother
 adding new syntax for the dozen remaining symbols from 2-3 modules that
 are commonly used in template constraints?


I understand. It is my humble opinion that we are "mostly pregnant" for 
as long as we require top-level imports. The real change of phase occurs 
when there are ZERO imports at top level. That's the prize DIP1005 is after.


 Arguing that local imports have been successful so we should simply do
 more of it is not a good argument, as there comes a point of diminishing
 returns.  You need to show that there are still worthile gains to be
 made from changing the language again, which is why I want to benchmark
 this feature with Walter before deciding.


You can be reasonably certain the benchmarks will improve as projected - 
starting in proportion to the the transitive fanout of top-level imports 
(10.5x for the standard library) and going down thereafter with the 
size, complexity, and actual dependencies of the module being compiled. 
All in all measurable but not dramatic. The same improvement will be 
brought about by lazy imports, and it won't be the deal maker/breaker. 
If you're waiting for the numbers to get convinced of anything, you 
already consider DIP1005 useless.


 Allow me to make an appeal regarding the review of any DIP. There
 seems to be a tendency of some reviewers to get attached and
 emotionally invested to their opinion, to the extent they'd be hurt by
 being "wrong" and would go to great lengths to argue they're "right".
 This has obvious negative effects on the entire process. Please don't.
 There's no loss of face to worry about. The only commitment we all
 should have is to the good of the D language. If DIP1005 reaches the
 conclusion of its own uselessness, I'd be the first one to write it up
 and close the PR.


 We could level this analysis back at you: you consider this DIP so
 "obvious" that you are not engaging with our concerns, making flip,
 incorrect remarks about how we would have bikeshedded local imports also.


I apologize it my remarks seem flippant, though I honestly have 
difficulty finding evidence of that. All I did was note that many of the 
arguments pitching lazy imports as a better solution than DIP1005 
(except of course for the Amdahl one) apply directly to local imports.


 In the end, this is a minor DIP that is easily bikeshedded, as everybody
 can grasp it and have an opinion on it.  I have refrained from
 commenting recently because I will let benchmarking settle it for me.
 Obviously, that won't suffice for others.


I do agree that if framed as a modest improvement in build economics, it 
is quite unimportant. But that's a problem with the DIP; its main 
strength is better encapsulation, which is no small thing.


Andrei
Jan 03 2017
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 01/03/2017 03:07 PM, Andrei Alexandrescu wrote:

 If you're waiting for the numbers to get convinced of anything, you
 already consider DIP1005 useless.


Eh, sorry, this came out wrong - as if I'm trying to put words in 
someone's mouth. I meant to say: "If you're waiting for the numbers to 
get convinced of anything, they won't be overwhelming so you may already 
write off DIP1005 as useless." -- Andrei
Jan 03 2017
prev sibling parent Joakim <dlang joakim.fea.st> writes:
On Tuesday, 3 January 2017 at 20:07:59 UTC, Andrei Alexandrescu 
wrote:

 Arguing that local imports have been successful so we should 
 simply do
 more of it is not a good argument, as there comes a point of 
 diminishing
 returns.  You need to show that there are still worthile gains 
 to be
 made from changing the language again, which is why I want to 
 benchmark
 this feature with Walter before deciding.


 You can be reasonably certain the benchmarks will improve as 
 projected - starting in proportion to the the transitive fanout 
 of top-level imports (10.5x for the standard library) and going 
 down thereafter with the size, complexity, and actual 
 dependencies of the module being compiled. All in all 
 measurable but not dramatic. The same improvement will be 
 brought about by lazy imports, and it won't be the deal 
 maker/breaker. If you're waiting for the numbers to get 
 convinced of anything, you already consider DIP1005 useless.


I don't need dramatic, a solid win is fine. A small 10% or less 
improvement in the speed of the import phase of compilation 
probably isn't worth it.


 Rather, the
 argument is that local imports mostly solved this problem, so 
 why bother
 adding new syntax for the dozen remaining symbols from 2-3 
 modules that
 are commonly used in template constraints?


 I understand. It is my humble opinion that we are "mostly 
 pregnant" for as long as we require top-level imports. The real 
 change of phase occurs when there are ZERO imports at top 
 level. That's the prize DIP1005 is after.


If you believe the benchmarks won't be dramatically different, I 
wonder why you think there will be a phase change.


 In the end, this is a minor DIP that is easily bikeshedded, as 
 everybody
 can grasp it and have an opinion on it.  I have refrained from
 commenting recently because I will let benchmarking settle it 
 for me.
 Obviously, that won't suffice for others.


 I do agree that if framed as a modest improvement in build 
 economics, it is quite unimportant. But that's a problem with 
 the DIP; its main strength is better encapsulation, which is no 
 small thing.


Encapsulation is not an end in itself.  You have listed four main 
advantages of this change in the DIP.  I have noted that I don't 
find the first three relating to reasoning and refactoring worth 
adding syntax for, while admitting that may be subjective.  The 
fourth talks about scalability, which I interpreted as either 
compilation speed or some other measurable gain, hence my asking 
for benchmarks.

If scalability refers to something else- it can't be reasoning or 
refactoring as those are listed as separate advantages- I'd like 
to hear what it is.  If it's measurable, I'd like to measure it 
first.
Jan 05 2017
prev sibling next sibling parent reply Paulo Pinto <pjmlp progrools.org> writes:
On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu 
wrote:

 On 12/31/2016 12:20 PM, Martin Nowak wrote:

 On Saturday, 24 December 2016 at 10:54:08 UTC, Stefan Koch 
 wrote:

 If that were made more lazy, we could import half of the 
 world with
 noticing impact.

 (Which espcially in std.traits, would not make that much of a
 difference since every template in there depends on nearly 
 every other
 template in there)


 Also the established technique of serializing precompiled AST 
 (after
 semantic3) of modules to a cache should be applicable as well.

 Cross-posting from
 https://github.com/dlang/DIPs/pull/51#issuecomment-269107966, 
 b/c it
 wasn't answered yet.


 No worries, I'll make a pass (a rewrite, really) taking all 
 feedback into account.


 Were any other means considered? This is proposing to add 
 plenty of
 additional annotations only to speed up compilation, but none 
 of the
 classical tools for pre-compilation were assessed.
 Since D's modules don't have the header problem, even 
 pre-compilation
 and reuse of semantic3 should be possible, or not?


 DIP1005 gives consideration to the speed of compilation aspect 
 in larger proportion than speed's importance; the first and 
 foremost benefit of DIP1005 is it closes the gap on dependency 
 encapsulation, which had been very successfully narrowed by 
 local imports. However, the DIP will keep the experiments and 
 results on speed measurements because they are relevant to it 
 and any related methods of lazily loading of modules.

 Regarding the ongoing doubts about the advantages of inline 
 imports: they are first and foremost a completion of the nested 
 import feature. As such, most, if not all, arguments against 
 inline imports apply equally to nested imports. Come to think 
 of it, lazy imports vs nested imports:

 * same improvement in compilation speed? check
 * no language changes? check
 * no nasty bugs in the aftermath (such as the infamous 
 https://issues.dlang.org/show_bug.cgi?id=10378)? check
 * scalable builds? check

 Yet local imports are overwhelmingly superior to lazy imports 
 because of one thing: they localize dependencies. They 
 introduce modularity and its ancillary perks (fast and scalable 
 builds, easier review and refactoring) not by engineering, but 
 by organically placing dependencies spatially with their 
 dependents. (The scope statement does the same thing with 
 temporal dependencies.) That the DIP does not make it clear 
 that it is a necessary and sufficient extension of local 
 imports is a problem with it.

 I now am really glad we slipped local imports before the 
 formalization of DIPs. The feature could have been easily 
 demeaned out of existence.

 Allow me to make an appeal regarding the review of any DIP. 
 There seems to be a tendency of some reviewers to get attached 
 and emotionally invested to their opinion, to the extent they'd 
 be hurt by being "wrong" and would go to great lengths to argue 
 they're "right". This has obvious negative effects on the 
 entire process. Please don't. There's no loss of face to worry 
 about. The only commitment we all should have is to the good of 
 the D language. If DIP1005 reaches the conclusion of its own 
 uselessness, I'd be the first one to write it up and close the 
 PR.


 Thanks,

 Andrei


Allow me just to share a worthless outsider opinion.

I never contributed anything worthwhile and decided it was better 
to just focus on JVM, .NET languages., alongside C++, as those 
are the skills I get paid for, thus stop polluting D forums.

Looking from the outside, and watching what was reached from 2016 
roadmap, it is clear the DIPs evaluated thus dar aren't about 
fixing the library or runtime issues that prevent D's adoption at 
large as a systems programming language.

Meanwhile Swift, Go and Rust have a clear roadmap how their 
future is supposed to look like, and drive just in that 
direction, with C++ taking all remaining good D ideas.

This DIP discussion and the latest ones about splitting the 
runtime again, don't do anything to earn D any credibility it 
might still have left.
Jan 03 2017
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 01/03/2017 04:28 AM, Paulo Pinto wrote:

 Looking from the outside, and watching what was reached from 2016
 roadmap, it is clear the DIPs evaluated thus dar aren't about fixing the
 library or runtime issues that prevent D's adoption at large as a
 systems programming language.

 Meanwhile Swift, Go and Rust have a clear roadmap how their future is
 supposed to look like, and drive just in that direction, with C++ taking
 all remaining good D ideas.

 This DIP discussion and the latest ones about splitting the runtime
 again, don't do anything to earn D any credibility it might still have
 left.


Thanks for taking the time to write this. All languages have an 
improvement process, which runs as a background task that is not always 
correlated with the current principal objectives of the language's 
leadership. We have recently improved our DIP process, but positive 
examples of compelling DIPs are missing. We were facing the problem that 
upcoming DIPs, if unguided, could create busywork for the leadership on 
trivial matters. Whether approved, rejected, or put on hold, DIP1005 
will serve as a yardstick for future DIPs regarding the necessary work 
involved in defining the problem, investigating it as rigorously as 
possible, comparing with alternatives, and defining the feature.

Walter and I do agree that the DIP is ahead of its time because (a) 
there are more pressing and more important matters to keep me busy, and 
(b) encapsulation in D is a larger topic that we haven't yet put front 
and center because of (a).

For DIP1005 I invite interested collaborators to contact me about future 
iterations; it is important for the matter of encapsulation and also for 
the meta-challenge of setting an example for other DIPs to follow. Thanks.


Andrei
Jan 03 2017
prev sibling parent Joakim <dlang joakim.fea.st> writes:
On Tuesday, 3 January 2017 at 09:28:06 UTC, Paulo Pinto wrote:

 Allow me just to share a worthless outsider opinion.

 I never contributed anything worthwhile and decided it was 
 better to just focus on JVM, .NET languages., alongside C++, as 
 those are the skills I get paid for, thus stop polluting D 
 forums.

 Looking from the outside, and watching what was reached from 
 2016 roadmap, it is clear the DIPs evaluated thus dar aren't 
 about fixing the library or runtime issues that prevent D's 
 adoption at large as a systems programming language.


Without listing what you think those real issues are, such a 
pronouncement is useless.


 Meanwhile Swift, Go and Rust have a clear roadmap how their 
 future is supposed to look like, and drive just in that 
 direction, with C++ taking all remaining good D ideas.


Heading in the wrong direction at full speed ahead is worse than 
meandering along in no discernible direction, because worst case, 
you will likely be closer to the right direction after simply 
meandering.  In any case, you clearly think D headed in some good 
directions, or C++ would have nothing to copy. ;)

As for the other roadmaps, I see it for Rust and Swift

https://github.com/aturon/rfcs/blob/roadmap-2017/text/0000-roadmap-2017.md
https://github.com/apple/swift-evolution

but nothing for Go.  Do you mean these milestone lists with 
hundreds of PRs attached?

https://github.com/golang/go/milestones

Or is it somewhere else that google can't easily find it?  You 
could read the changelog for the dmd/phobos master branches and 
get the same result as that Go list.

Those languages are all backed by large tech companies and are 
driven by their company agendas.  As I've pointed out in this 
forum before, with some quotes from Linus 
(http://forum.dlang.org/post/dluoruxmwxnfjtyvmgbh forum.dlang.org), directed
development is good for niches, which is where all those languages are now
(iOS, network services, Servo), but only general-purpose tech lasts in the
medium- to long-term.

D may never gain the resources it needs to make it that long, but 
a community employing D in _no one direction_, as Linus says 
happened with linux, is more likely to make it general-purpose 
enough to survive.


 This DIP discussion and the latest ones about splitting the 
 runtime again, don't do anything to earn D any credibility it 
 might still have left.


I kind of agree that both are not that worthwhile, but without 
stating your reasons for thinking the discussion _alone_ hurts 
D's credibility, which I mostly disagree with as such discussion 
is a core part of the distributed OSS process, that opinion is 
again useless.
Jan 04 2017
prev sibling next sibling parent reply Daniel N <no public.email> writes:
On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu 
wrote:

 No worries, I'll make a pass (a rewrite, really) taking all 
 feedback into account.


Are all the proposals mutually exclusive? When it comes to 
declarations, the DIP actually won me over! However I was hoping 
for it to solve a different issue inside the body with local 
imports. I tend to write short helper functions, with the result 
that in the common case, an imported symbol is often used only 
once per function.

Currently I consider it good style to list all imports in the 
beginning of a scope, but based on past experience with C89, this 
is not entirely optimal, however Timon Gehr's proposal could 
solve it.

import and directly invoke:
import std.traits : isArray(...)

=======================================================
[Proposed Style - analogous to C99]
body
{
   static if(...)
     import std.range : zip(...);
   static if(import std.traits : isArray(...)))
     import std.range : join(...);
}
=======================================================
[Old Style - analogous to C89 - Suffers from "unused" imports]
body
{
   import std.range : zip, join;
   import std.traits : isArray;

   static if(...)
     zip(...)
   static if(isArray(...)))
     join(...)
}
=======================================================
[Alternate Style - Suffers from DRY and can't limit scope of 
isArray]
body
{
   import std.traits : isArray;

   static if(...)
   {
     import std.range : zip;
     zip(...);
   }
   static if(is(isArray(...)))
   {
     import std.range : join;
     join(...);
   }
}

C89 vs C99 style I was referring to if it wasn't clear.
=======================================================
[C89]
void foo(void)
{
#ifdef ... /* needed to silence unused variable warnings */
   int x;
#endif
... lots of code ...
#ifdef ...
   x = bar();
#endif
}

[C99]
void foo(void)
{
#ifdef ...
   int x = bar();
#endif
}
=======================================================
Jan 03 2017
parent Daniel N <no public.email> writes:
On Tuesday, 3 January 2017 at 10:18:57 UTC, Daniel N wrote:

 static if(is(isArray(...)))
 {
   import std.range : join;
   join(...);
 }


PS Actually double {} is needed for the 'alternate' style to be 
meaningful, preventing the import from bleeding into the outer 
scope.

static if(isArray(...))
{{
   import std.range : join;
   join(...);
}}
Jan 03 2017
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
There are quite a few fallacies in there.

On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu 
wrote:

 Regarding the ongoing doubts about the advantages of inline 
 imports: they are first and foremost a completion of the nested 
 import feature. As such, most, if not all, arguments against 
 inline imports apply equally to nested imports. Come to think 
 of it, lazy imports vs nested imports:

 * same improvement in compilation speed? check
 * no language changes? check
 * no nasty bugs in the aftermath (such as the infamous 
 https://issues.dlang.org/show_bug.cgi?id=10378)? check
 * scalable builds? check

 Yet local imports are overwhelmingly superior to lazy imports 
 because of one thing: they localize dependencies. They 
 introduce modularity and its ancillary perks (fast and scalable 
 builds, easier review and refactoring) not by engineering, but 
 by organically placing dependencies spatially with their 
 dependents. (The scope statement does the same thing with 
 temporal dependencies.) That the DIP does not make it clear 
 that it is a necessary and sufficient extension of local 
 imports is a problem with it.


There is a major difference with this DIP.

Lazy import is not a language change, but a compiler 
implementation detail. As such, it doesn't require a DIP or 
anything specific.

Nested import are a language simplification. Declaration can 
appear anywhere, import is a declaration, the fact that import 
couldn't appear anywhere was an arbitrary limitation, and 
removing it makes the language simpler. As such, the burden of 
proof is on maintaining the limitation rather than removing it.

This DIP is a language addition. Therefore, contrary to nested or 
lazy import, the burden of proof is on it. This DIP should be 
considered as follow: how much complexity does it add and how 
much benefit does it bring, compared to alternatives.

The obvious benefit is localizing dependencies. I think I'm not 
too far off track by considering most of the speedup and scalable 
build can be achieved with lazy import and, while I'm sure there 
are example where this is superior, we are talking marginal gains 
as lazy and nested imports squeezed most of the juice already.

The cost is the language addition. The first obvious improvement 
that can be made to this DIP to reduce its cost is to not 
introduce a new syntax. As such, the addition is limited to 
allowing the existing syntax in a new place rather than adding a 
whole new syntax for imports.

I like the extra expressivity. I'm not 100% convinced it is worth 
the extra cost, but the more the cost is reduced, the more 
rational it seems to me that this option should be pursued.


 I now am really glad we slipped local imports before the 
 formalization of DIPs. The feature could have been easily 
 demeaned out of existence.


Good you also notice how broken the DIP process is.

One suggestion: let's keep the DIP describing the change to be 
made. Some examples are fine to illustrate, but it is not the 
DIp's purpose to be easy to understand or expand too much in 
argumentation, or it'll be useless as a spec document, and trying 
to have the DIP be a spec, a tutorial, a essay on why the 
feature, and so on just lead to endless rewriting lead to 
perpetual motion but no progress.
Jan 04 2017
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 04.01.2017 16:00, deadalnix wrote:

 Nested import are a language simplification. Declaration can appear
 anywhere, import is a declaration, the fact that import couldn't appear
 anywhere was an arbitrary limitation, and removing it makes the language
 simpler. As such, the burden of proof is on maintaining the limitation
 rather than removing it.
 ...


I don't fully agree. Nested imports, the way they have been implemented, 
pose a new symbol hijacking hazard.

(A symbol hijacking anecdote (not directly related): 
https://github.com/tgehr/d-compiler/pull/1#discussion-diff-89697186L85 )


 This DIP is a language addition. Therefore, contrary to nested or lazy
 import, the burden of proof is on it. This DIP should be considered as
 follow: how much complexity does it add and how much benefit does it
 bring, compared to alternatives.


It adds basically no implementation complexity [1]. I consider the 
benefit real, but minor enough to oppose the DIP based on its wacky syntax.


[1] Both static if and static foreach (once it lands) need the same kind 
of scoping rules.
Jan 04 2017
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 4 January 2017 at 15:56:13 UTC, Timon Gehr wrote:


 [1] Both static if and static foreach (once it lands) need the 
 same kind of scoping rules.


Please do contact me if you are working on static foreach, there 
are dmd and implementation specific issues to be taken into 
account.
Jan 04 2017
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Wednesday, 4 January 2017 at 16:03:29 UTC, Stefan Koch wrote:

 On Wednesday, 4 January 2017 at 15:56:13 UTC, Timon Gehr wrote:


 [1] Both static if and static foreach (once it lands) need the 
 same kind of scoping rules.


 Please do contact me if you are working on static foreach, 
 there are dmd and implementation specific issues to be taken 
 into account.


I think the best path forward is to define them properly.
Jan 04 2017
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 04.01.2017 17:03, Stefan Koch wrote:

 On Wednesday, 4 January 2017 at 15:56:13 UTC, Timon Gehr wrote:


 [1] Both static if and static foreach (once it lands) need the same
 kind of scoping rules.


 Please do contact me if you are working on static foreach,


Not currently. I might implement it in 
https://github.com/tgehr/d-compiler soon though. I'll let you know.


 there are dmd and implementation specific issues to be taken into account.


What kind of issues? The compiler implementation shouldn't matter for 
the definition of the static foreach behaviour.

Generation of the sequence of values assigned to the loop variables 
should be exactly like runtime foreach, scoping of the loop variables 
should be local. For scoping of the declarations within the static 
foreach body, there are multiple possibilities. (The most obvious thing 
is to just insert them into the outer scope, but then we need some other 
improvements in order to avoid having to string mixin each and every 
declaration in the body.)

There are some questions about which dependency structures should be 
allowed for the loop aggregate and the declarations in the loop body, 
but I think this is mostly orthogonal to static foreach itself.
Jan 04 2017
prev sibling parent deadalnix <deadalnix gmail.com> writes:
On Wednesday, 4 January 2017 at 15:56:13 UTC, Timon Gehr wrote:

 I don't fully agree. Nested imports, the way they have been 
 implemented, pose a new symbol hijacking hazard.


I'd argue this was an existing bug in import handling. This is 
why I like to have very orthogonal definitions.


 It adds basically no implementation complexity [1]. I consider 
 the benefit real, but minor enough to oppose the DIP based on 
 its wacky syntax.

 [1] Both static if and static foreach (once it lands) need the 
 same kind of scoping rules.


I know about [1], this is why I did not mentioned it. I don't 
really mind about implementation complexity, I care about 
complexity of the definition. For the following reasons:
  - If the implementation may be complex, it can be isolated 
and/or abstracted away.
  - Interaction with other parts of the language are more 
predictable, including future parts that do not exists yet.
  - It obviate the explosion of trivia experienced devs needs to 
know to use the language.
Jan 04 2017
prev sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Sat, 24 Dec 2016 04:34:03 -0500, Andrei Alexandrescu wrote:

 Upon more investigation, I see a large discrepancy between the findings
 of DIP1005 and yours.


There's no discrepancy.

In part, you are misinterpreting most of what I said.

In part, you are assuming that imports on non-template declarations will 
be handled lazily, even though that is not part of this DIP, even though 
that is likewise possible with static and selective imports.

In part, you are using lines of code as a proxy for compile time.

In part, you dispute that this only affects template constraints, but:
* An import used only in the body of a template can be made a local 
import today.
* An import used in the declaration of a templated type or function can 
be addressed by using explicit template syntax, offering a place to 
insert your imports.
* An import used anywhere else must still be processed, even assuming 
this DIP is implemented.
* If, in a future DIP, we make it so that `with(import)` is handled 
lazily, we can also make it so that static and selective imports are 
handled lazily.


 The findings of DIP1005 are the following:
 
 * Importing a single std module also imports on average 10.5 other
 modules.


Seems reasonable. Between 2 and 3.5 direct dependencies, by my count, and 
you're counting transitive dependencies.

We're concerned with the effects of DIP1005, though, which only affects 
template constraints.


 * Importing a single std module costs on average 64.6 ms.


55-ish for your hardware, you reported elsewhere. 47-ish for mine.


 * (Not stated in the DIP) A majority of std templates would acquire
 inline imports.


Again, that wouldn't impact compile times because these aren't template 
constraints.

You can make a separate DIP to make imports lazy. That can impact static, 
selective, and `with` imports equally well. But it's not part of what 
we're discussing today.


 According to the DIP, one may estimate that the proposed feature would
 reduce additional imports to 0 and the average time to import a single
 module by a factor of 10 to under 10 ms.


"The proposed feature" must be lazy semantic analysis, especially of 
imports. That isn't part of DIP1005.

You won't get to zero additional imports. You might get to zero 
*extraneous* imports -- that is, only the set of imports required to 
create a custom *.di file containing only the parts of the module that 
your application uses.


 By your estimates:
 
 * 26 templates in std need inline imports.


I said that 26 templates *could possibly benefit from* your new style of 
imports. There's a difference between possibly benefitting from a change 
and needing that change.


 * Importing a single std module today would only imports 1-3 other
 modules most of the time (one or more of std.traits, std.meta, and
 std.range.primitives).


No, that's not what I said at all. I said that the only modules you would 
sometimes *stop* processing because of DIP1005 are std.traits, std.meta, 
and std.range.primitives. That's because those modules contain templates 
used in other modules as template constraints.

In order to get any additional improvements, you need lazy imports, which 
can also apply to static or selective imports without any syntax changes.


 * These additional imports cost in aggregate under 10ms, bringing the
 average cost of importing a module itself to 54.6 ms.


~10ms is the upper bound of the added cost if you import just one module 
in std that has a template constraint you don't use.

The way you state it implies that every module brings in std.traits, 
std.meta, and std.range.primitives unnecessarily, instead of 26 templates 
across at most 26 modules importing them for a reason.


 * It follows that the average module takes 5.46 more times to import
 alone than the sum of std.traits, std.meta, and std.range.primitives
 (which have a total of 11263 lines, 5x more than the average Phobos
 module).


More like 4.7 on my hardware, but yeah. 11k lines that have to be parsed 
and 0 lines that require semantic analysis. Not terribly surprising.


 I don't see how your claims can be simultaneously true with the findings
 of DIP1005.


You found that the average cost of importing a std module is 54ms or 
thereabouts. std.traits, std.meta, and std.range.primitives are well 
below average. No conflict there. They aren't even the cheapest modules 
in the standard library.

The modules in question are mostly unittests. The compiler doesn't run 
semantic on unittests in a module that wasn't included in the command 
line. (Even if you pass -unittest. Try it out -- you can even have a 
unittest that says `static assert(false);` and it does nothing.)

The parts of the modules that are not unittests are templates. The 
compiler doesn't run semantic analysis on templates until you use them.

So it should be pretty obvious why these modules are so cheap to import 
and not use.


 The scripts that compute those numbers are available with
 the DIP. Were you able to reproduce them?


The times it reported on my hardware:

Min: 5ms
Max: 300ms
Median: 21ms
Average: 47ms

The minimum isn't terribly useful because it gets to the point of testing 
the process scheduler and IO more than the compiler. If we want numbers 
that we can trust on the low end, we'll need to put timing information 
into the compiler, maybe control for IO by using a ramfs, that sort of 
thing.

You also reproduced my test, so this isn't a quirk of my installation.
Dec 24 2016
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 24 December 2016 at 17:52:04 UTC, Chris Wright wrote:

 The minimum isn't terribly useful because it gets to the point 
 of testing the process scheduler and IO more than the compiler. 
 If we want numbers that we can trust on the low end, we'll need 
 to put timing information into the compiler, maybe control for 
 IO by using a ramfs, that sort of thing.


As a good approximation you can use a profile build of dmd.
It will give you a breakdown of how long the individual functions 
took.
Due to the way the profiling works, it does homogenize the values 
a bit.
Files are not streamed into dmd, it reads them as one Block.
Therefore there is no need for a ramfs.
Dec 24 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/24/2016 12:52 PM, Chris Wright wrote:

 On Sat, 24 Dec 2016 04:34:03 -0500, Andrei Alexandrescu wrote:

 Upon more investigation, I see a large discrepancy between the findings
 of DIP1005 and yours.


 There's no discrepancy.

 In part, you are misinterpreting most of what I said.


Yah, I suspected so.

* Your post provided no description of the methodology used beyond 
"relatively simple regex" to come up with the magic number 26, so I 
tried to use interpretation.

* There is no answer to "where is cmp on that list for example?" to shed 
light on what 26 is about.

* I have no access to your method of measurement, the scripts you used, 
or really what the claims are.

So I really don't have a good understanding of what hypothesis you have 
and how you arrived to it. DIP1005 carefully describes methodology and 
publishes measurements and results (which I'd be in your debt if you 
used - not those "published elsewhere"). Then it interprets the results.


 In part, you are assuming that imports on non-template declarations will
 be handled lazily, even though that is not part of this DIP, even though
 that is likewise possible with static and selective imports.


Imports in non-template declarations ARE handled lazily when those 
non-templates are imported. Consider:

module test;
void fun()
{
     import std.stdio;
     writeln("hello");
}

module test2;
import test;
void main() {} // let's not call fun

dmd -v test2.d

====
binary    dmd
version   v2.073.0-devel-bd0dec2
config    /home/andrei/bin/dmd.conf
parse     test2
importall test2
import    object	(/home/andrei/d/druntime/import/object.d)
import    test	(test.d)
semantic  test2
entry     main      	test2.d
semantic2 test2
semantic3 test2
code      test2
function  D main
cc test2.o -o test2 -m64 
-L/home/andrei/d/phobos/generated/linux/release/64 -Xlinker -Bstatic 
-lphobos2 -Xlinker -Bdynamic -lpthread -lm -lrt -ldl
====

Destroyed? This might be another misinterpretation of what you said though.


 In part, you are using lines of code as a proxy for compile time.


What do you suggest to use?


 In part, you dispute that this only affects template constraints, but:
 * An import used only in the body of a template can be made a local
 import today.


Correct. To the best of my knowledge, that fruit has been picked in Phobos.


 * An import used in the declaration of a templated type or function can
 be addressed by using explicit template syntax, offering a place to
 insert your imports.


Good thought. Would be a bit overkill though.


 * An import used anywhere else must still be processed, even assuming
 this DIP is implemented.


How do you mean that? On the face of it the sentence is true. Are you 
saying that you need to remove top-level imports to benefit from the 
DIP? That would indeed be the case.


 * If, in a future DIP, we make it so that `with(import)` is handled
 lazily, we can also make it so that static and selective imports are
 handled lazily.


That would be nice, but DIP1005 provides advantages beyond latency.

BTW it would be great if the discussion focused on (a) things that are 
true but the DIP doesn't mention (or misrepresents), or (b) things that 
are false that the DIP claims. I clearly concede that the DIP cannot 
convince someone with enough cognitive bias without an implementation 
and a few years of experience (and even then - aren't there folks out 
there who believe things like constraints, static if, or ranges are 
failures?).


 The findings of DIP1005 are the following:

 * Importing a single std module also imports on average 10.5 other
 modules.


 Seems reasonable. Between 2 and 3.5 direct dependencies, by my count, and
 you're counting transitive dependencies.


Shouldn't I? Shouldn't you?


 We're concerned with the effects of DIP1005, though, which only affects
 template constraints.


 * Importing a single std module costs on average 64.6 ms.


 55-ish for your hardware, you reported elsewhere. 47-ish for mine.


Let's stick with the numbers published in the DIP.


 * (Not stated in the DIP) A majority of std templates would acquire
 inline imports.


 Again, that wouldn't impact compile times because these aren't template
 constraints.


I don't understand this. YES it would impact compile times!


 You can make a separate DIP to make imports lazy. That can impact static,
 selective, and `with` imports equally well. But it's not part of what
 we're discussing today.


 According to the DIP, one may estimate that the proposed feature would
 reduce additional imports to 0 and the average time to import a single
 module by a factor of 10 to under 10 ms.


 "The proposed feature" must be lazy semantic analysis, especially of
 imports. That isn't part of DIP1005.


The proposed feature is DIP1005. I really don't understand how your 
discourse goes.


 You won't get to zero additional imports. You might get to zero
 *extraneous* imports -- that is, only the set of imports required to
 create a custom *.di file containing only the parts of the module that
 your application uses.


You are wrong. With no top-level imports and all imports hoisted into 
inline imports, the fixed cost of importing one module will be that 
module alone. To focus the discussion, could we please focus on this one 
point?


Andrei
Dec 24 2016
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/24/2016 02:03 PM, Andrei Alexandrescu wrote:

 With no top-level imports and all imports hoisted into inline imports,
 the fixed cost of importing one module will be that module alone.


I'd really like to clarify this because that's where scalability comes 
from. The moment you need to open an unbounded amount of files for one 
import, it's gone. If I missed something and more than one module needs 
to be looked at I need to rethink the whole DIP. -- Andrei
Dec 24 2016
prev sibling parent Chris Wright <dhasenan gmail.com> writes:
On Sat, 24 Dec 2016 14:03:26 -0500, Andrei Alexandrescu wrote:

 * Your post provided no description of the methodology used beyond
 "relatively simple regex" to come up with the magic number 26, so I
 tried to use interpretation.
 
 * There is no answer to "where is cmp on that list for example?" to shed
 light on what 26 is about.


You mentioned that in another branch of the conversation. It took some 
time for me to get to it. I'm a human.


 In part, you are assuming that imports on non-template declarations
 will be handled lazily, even though that is not part of this DIP, even
 though that is likewise possible with static and selective imports.


 
 Imports in non-template declarations ARE handled lazily when those
 non-templates are imported. Consider:


...

Y'know, I make assumptions, but I tend to test them. But that kind of 
depends on me writing valid tests, and in this case, I didn't.

My face is egged, and I concede.

Thank you for providing example code. That cleared everything up very 
fast.

With this new understanding, most modules become nearly free to import 
under your proposal. It does require a lot of work to get there, but on 
the plus side, it should be possible to automate that with a tool. I'd be 
willing to add that to my build process for anything I publish if it'll 
help others.


 In part, you are using lines of code as a proxy for compile time.


 
 What do you suggest to use?


You did write a script to check compile times for importing individual 
phobos modules.


 The findings of DIP1005 are the following:

 * Importing a single std module also imports on average 10.5 other
 modules.


 Seems reasonable. Between 2 and 3.5 direct dependencies, by my count,
 and you're counting transitive dependencies.


 
 Shouldn't I? Shouldn't you?


I was agreeing with you. At the time, I didn't have a convenient way to 
check the number of transitive dependencies of a module.


 We're concerned with the effects of DIP1005, though, which only affects
 template constraints.


 * Importing a single std module costs on average 64.6 ms.


 55-ish for your hardware, you reported elsewhere. 47-ish for mine.


 
 Let's stick with the numbers published in the DIP.


Sure. You insinuated that I should run your script to verify your 
numbers, so I did.
Dec 24 2016
prev sibling next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu 
wrote:

 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


FYI, from the DIP, this is false:

"The static import setup does not share the issue above, at the 
cost of being cumbersome to use---all imported symbols must use 
full lookup everywhere. A reasonable engineering approach would 
be to define shorter names:"

It then proceeds to list a bunch of local aliases. That's not how 
I'd do it because you can also simply use a renamed import:

static import r = std.range.primitives;

void foo(R)(R range) if(r.isInputRange!(range)) // works today


If the import were lazy - which there's no reason not to be with 
a static import, renamed or not, since its use is always explicit 
- then this gives most the same advantages of the DIP.

Granted, you'd still have to look up what `r` means, so it isn't 
100% dependency carrying.



Another current language option not discussed in the document is 
an eponymous template.


But there is another language option today: an eponymous template.

Consider the following:

with (import std.stdio) void process(File input) ;

Let's rewrite that to be:

template process() {
    import std.stdio;
    void process(File input) {

    }
}


The whole symbol is now lazy and encapsulated... and thanks to 
the existing eponymous rules, it'd be semi-transparent to the 
user (just error messages would be the main source of leakage, oh 
god the error messages could be so ugly).

But it works in today's D.



BTW it is my opinion that the "one file, one module" rule is a 
mistake. Even with all these things, the declarations are still 
not *guaranteed* to carry their dependencies, since top-level 
imports still leak in. Separate modules don't have that problem, 
and if we could just define several modules in one file, we'd 
basically destroy this DIP in one swift stroke.

I doubt y'all would change that, but still, I'd list that as an 
alternative in the section that lists many small files as being a 
major disadvantage, since while we are discussing changing the 
language, that's a possible change too unless explicitly off the 
table.
Dec 31 2016
next sibling parent Chris Wright <dhasenan gmail.com> writes:
On Sat, 31 Dec 2016 15:05:00 +0000, Adam D. Ruppe wrote:

 static import r = std.range.primitives;


For that to pass code review, you'd need a readable name -- perhaps 
'ranges' instead of 'r' -- which makes it more obvious, more searchable, 
and more typing.


 with (import std.stdio) void process(File input) ;
 
 Let's rewrite that to be:
 
 template process() {
     import std.stdio;
     void process(File input) {
 
     }
 }


But that's a lot of typing, so nobody's going to do it. Just like how we 
use normal imports rather than static or selective imports everywhere. 
It's even more verbose than DIP1005.


 BTW it is my opinion that the "one file, one module" rule is a mistake.
 Even with all these things, the declarations are still not *guaranteed*
 to carry their dependencies, since top-level imports still leak in.
 Separate modules don't have that problem, and if we could just define
 several modules in one file, we'd basically destroy this DIP in one
 swift stroke.


That's rather elegant, though I don't see how you'd import a module in a 
file that defines several modules. If they were anonymous and importing 
the file gave you access to all of them, that makes sense. Otherwise it's 
tricky to figure out which sub-file modules exist.
Dec 31 2016
prev sibling parent Meta <jared771 gmail.com> writes:
On Saturday, 31 December 2016 at 15:05:00 UTC, Adam D. Ruppe 
wrote:

 Consider the following:

 with (import std.stdio) void process(File input) ;

 Let's rewrite that to be:

 template process() {
    import std.stdio;
    void process(File input) {

    }
 }


I absolutely hate making functions templated if they don't have 
to be. It breaks a whole bunch of things and is not at all 
transparent to the user. You can't take its address, pass it as a 
delegate/function pointer, use certain traits to introspect it 
(e.g. isSomeFunction, ReturnType, etc.), not to mention how 
templates really don't work well with inheritance. This all goes 
out the window when you turn a regular function into a template.
Dec 31 2016
prev sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Tue, 13 Dec 2016 17:33:24 -0500, Andrei Alexandrescu wrote:


 Destroy.
 
 https://github.com/dlang/DIPs/pull/51/files
 
 
 Andrei



 Inside any function, with (Import ImportList) is a statement that
 introduces a scope.


So I can't write:

  with (import std.stdio, std.conv)
  int count = readln().to!int;
  assert(count > 0);

It is in fact entirely equivalent to write:

  {
    import std.stdio, std.conv;
    int count = readln().to!int;
  }

I think I'd just put a line in the DIP: for consistency, this syntax 
works anywhere a with statement is currently allowed, but it's not 
recommended to use it inside function bodies in general.


 This extension removes an unforced limitation of the current with
 syntax (allows it to occur at top level)


In other words, another aspect of this DIP is that I can write:

  module foo;
  static import std.traits;
  static import bar;
  with (std.traits)
  {
    template Foo(T) if (isAbstractClass!T) {}
  }
  with (bar.SomeEnum)
  {
    enum something = SomeEnumValue;
  }

Which *almost*, but not quite, obviates the "you can put the import list 
in the with clause" part of the DIP. It's got all the same benefits for 
reading, but you might occasionally have to jump to the top of the file 
to add a new static import.
Dec 31 2016
parent reply pineapple <meapineapple gmail.com> writes:
On Saturday, 31 December 2016 at 17:02:55 UTC, Chris Wright wrote:

 This extension removes an unforced limitation of the current 
 with syntax (allows it to occur at top level)


 In other words, another aspect of this DIP is that I can write:

   module foo;
   static import std.traits;
   static import bar;
   with (std.traits)
   {
     template Foo(T) if (isAbstractClass!T) {}
   }
   with (bar.SomeEnum)
   {
     enum something = SomeEnumValue;
   }


This is the only expression of the feature I've seen so far that 
makes intuitive sense to me.

I'm still not sold on it being a worthy addition but, if it were, 
then this is the most promising syntax I've seen so far.
Jan 01 2017
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 02.01.2017 01:47, pineapple wrote:

 On Saturday, 31 December 2016 at 17:02:55 UTC, Chris Wright wrote:

 This extension removes an unforced limitation of the current with
 syntax (allows it to occur at top level)


 In other words, another aspect of this DIP is that I can write:

   module foo;
   static import std.traits;
   static import bar;
   with (std.traits)
   {
     template Foo(T) if (isAbstractClass!T) {}
   }
   with (bar.SomeEnum)
   {
     enum something = SomeEnumValue;
   }


 This is the only expression of the feature I've seen so far that makes
 intuitive sense to me.

 I'm still not sold on it being a worthy addition but, if it were, then
 this is the most promising syntax I've seen so far.


Should be 'static with'. ('with' already has a meaning.)
Jan 01 2017
prev sibling parent Chris Wright <dhasenan gmail.com> writes:
On Mon, 02 Jan 2017 00:47:41 +0000, pineapple wrote:


 On Saturday, 31 December 2016 at 17:02:55 UTC, Chris Wright wrote:

 This extension removes an unforced limitation of the current with
 syntax (allows it to occur at top level)


 In other words, another aspect of this DIP is that I can write:

   module foo;
   static import std.traits;
   static import bar;
   with (std.traits)
   {
     template Foo(T) if (isAbstractClass!T) {}
   }
   with (bar.SomeEnum)
   {
     enum something = SomeEnumValue;
   }


 
 This is the only expression of the feature I've seen so far that makes
 intuitive sense to me.
 
 I'm still not sold on it being a worthy addition but, if it were, then
 this is the most promising syntax I've seen so far.


This is a minor part of the proposal that's mentioned as an aside, 
though, and I'm not even sure it was even intentional.

This alone doesn't get the two benefits Andrei is most after -- 
specifically, that you can add the import in place without going to the 
top of the file and that you can move the declaration to another module 
without worrying about imports.
Jan 01 2017