www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Dlist and dip1000 challenge

reply Steven Schveighoffer <schveiguy gmail.com> writes:
I've implemented a mergesort for both slist and dlist in phobos. I found 
it easier to start with slist, since the algorithm and operation is 
basically the same, dlist will simply have a post-sort operation that 
reconnects all the prev pointers.

I went to apply this to dlist, and found a disturbing thing -- all 
accesses to data in dlist are done via *casting*. The idea behind it is, 
we only store non-templated nodes inside a list, and those nodes only 
have prev and next pointers. Then you cast to the correct "PayNode" 
type, giving you the actual data. Things are allocated properly, so it 
works, but holy crap, what a mess. For those who are interested, here is 
the original PR that introduced this: 
https://github.com/dlang/phobos/pull/2457.

I decided, I'll make a PR to undo all that (the savings from doing this 
can't be worth it, the only thing that's non-templated is the range 
popFront and popBack), but I've now run into an issue with dip1000. 
Apparently, some of the casting confuses the compiler sufficiently 
enough that it can't track lifetimes, and it just rubber-stamps it (at 
least, that's my theory).

I've removed the "BaseNode" type, and just use "PayNode" everywhere. All 
of a sudden, things become un- safe.

So I started slapping  safe tags on to see where the issue is. It starts 
with something I've known is an issue for a long time, and that is 
having a scoped array of strings (or other reference types). You get 
this with a type-safe variadic function.

The compiler treats all the array items as if they were scope, meaning 
you can't assign a string, either allocated from the heap or a literal, 
to things that aren't scope, which doesn't make any sense. Sure the 
array is scope, but not what the elements point at!

In any case, I created a mock-up of the parts that are not working, can 
anyone find either an issue we can file for dip1000 or a way to solve 
this properly? I found I can mark stuff trusted, but I *really* don't 
like that. It's not much worse than casting everything, however.

Here is the mockup code:
https://run.dlang.io/is/6xDFnr

I've marked main  safe, and the problematic function  safe (if you don't 
mark the problematic function  safe, it just complains that main can't 
do anything).

Note that in reality everything here is perfectly safe, I'm not escaping 
anything.

---------

Begin rant

So, here is one other thing I want to say. This took me HOURS to find, 
and narrow down. Not because I don't understand the concepts behind 
dip1000, but because the compiler has fully inserted so many hidden 
scopes, I don't know what the actual code it's compiling is. One big 
problem I think with dip1000 is simply that it's nearly impossible to 
understand where the issues are. Like I said at the end of the post 
above, the result of allowing compiler inference of dip1000 is that your 
whole program is simply marked unsafe, and you have absolutely no idea 
where it is. You can't even guess, because scope just shows up where you 
never typed it. Given that you NEED this functionality on templates, 
it's going to result, IMO, in people either not using dip1000, or giving 
up and adding  trusted: to the top of their file. This is going to be 
horrible if we can't find a way to either require scope instead of 
inferring it in some cases, or create a way to diagnose where the 
blasted problem actually is. Maybe something to say "I expected this 
call to be  safe, why isn't it".

End rant.

-Steve
Oct 23 2018
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Tuesday, 23 October 2018 at 15:10:24 UTC, Steven Schveighoffer 
wrote:

 I've implemented a mergesort for both slist and dlist in 
 phobos. I found it easier to start with slist, since the 
 algorithm and operation is basically the same, dlist will 
 simply have a post-sort operation that reconnects all the prev 
 pointers.

 I went to apply this to dlist, and found a disturbing thing -- 
 all accesses to data in dlist are done via *casting*. The idea 
 behind it is, we only store non-templated nodes inside a list, 
 and those nodes only have prev and next pointers. Then you cast 
 to the correct "PayNode" type, giving you the actual data. 
 Things are allocated properly, so it works, but holy crap, what 
 a mess. For those who are interested, here is the original PR 
 that introduced this: https://github.com/dlang/phobos/pull/2457.

 I decided, I'll make a PR to undo all that (the savings from 
 doing this can't be worth it, the only thing that's 
 non-templated is the range popFront and popBack), but I've now 
 run into an issue with dip1000. Apparently, some of the casting 
 confuses the compiler sufficiently enough that it can't track 
 lifetimes, and it just rubber-stamps it (at least, that's my 
 theory).

 I've removed the "BaseNode" type, and just use "PayNode" 
 everywhere. All of a sudden, things become un- safe.

 So I started slapping  safe tags on to see where the issue is. 
 It starts with something I've known is an issue for a long 
 time, and that is having a scoped array of strings (or other 
 reference types). You get this with a type-safe variadic 
 function.

 The compiler treats all the array items as if they were scope, 
 meaning you can't assign a string, either allocated from the 
 heap or a literal, to things that aren't scope, which doesn't 
 make any sense. Sure the array is scope, but not what the 
 elements point at!

 In any case, I created a mock-up of the parts that are not 
 working, can anyone find either an issue we can file for 
 dip1000 or a way to solve this properly? I found I can mark 
 stuff trusted, but I *really* don't like that. It's not much 
 worse than casting everything, however.

 Here is the mockup code:
 https://run.dlang.io/is/6xDFnr

 I've marked main  safe, and the problematic function  safe (if 
 you don't mark the problematic function  safe, it just 
 complains that main can't do anything).

 Note that in reality everything here is perfectly safe, I'm not 
 escaping anything.


My hunch (having looked at the source) would be that this is 
because a "value" to the compiler starts out not scope and must 
be proved to be scope. In

tail.next = allocate(stuff.front, tail);

there is an inductive step the compiler would need to make to 
infer tail as scope and that goes against the requirement to 
prove the scopeness of tail (with  safe the goal is to not have 
false positives, false negatives, like this, are annoying but 
fine). Please do add a bugziliqa for this even if it proves to be 
intractable to solve. We can at least document what the compiler 
can't infer.


 So, here is one other thing I want to say. This took me HOURS 
 to find, and narrow down. Not because I don't understand the 
 concepts behind dip1000, but because the compiler has fully 
 inserted so many hidden scopes, I don't know what the actual 
 code it's compiling is. One big problem I think with dip1000 is 
 simply that it's nearly impossible to understand where the 
 issues are. Like I said at the end of the post above, the 
 result of allowing compiler inference of dip1000 is that your 
 whole program is simply marked unsafe, and you have absolutely 
 no idea where it is. You can't even guess, because scope just 
 shows up where you never typed it. Given that you NEED this 
 functionality on templates, it's going to result, IMO, in 
 people either not using dip1000, or giving up and adding 
  trusted: to the top of their file. This is going to be 
 horrible if we can't find a way to either require scope instead 
 of inferring it in some cases, or create a way to diagnose 
 where the blasted problem actually is. Maybe something to say 
 "I expected this call to be  safe, why isn't it".


I totally agree.

Plug for https://github.com/dlang/dlang.org/pull/2453
We really need to better document this stuff.
Oct 23 2018
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/23/18 1:18 PM, Nicholas Wilson wrote:

 My hunch (having looked at the source) would be that this is because a 
 "value" to the compiler starts out not scope and must be proved to be 
 scope. In
 
 tail.next = allocate(stuff.front, tail);
 
 there is an inductive step the compiler would need to make to infer tail 
 as scope and that goes against the requirement to prove the scopeness of 
 tail (with  safe the goal is to not have false positives, false 
 negatives, like this, are annoying but fine). Please do add a bugziliqa 
 for this even if it proves to be intractable to solve. We can at least 
 document what the compiler can't infer.


Except that there should be no pointers to scope data here -- the only 
thing that is scope really is the string[] pointer. When you extract a 
string from that, it shouldn't be scope.

If it is being inferred scope, how does one have a scope array of 
non-scope pointers?


 I totally agree.
 
 Plug for https://github.com/dlang/dlang.org/pull/2453
 We really need to better document this stuff.


Well, documentation is good, but if there is a bug or design flaw for 
dip1000 that's what we need to focus on.

Bottom line: I should be able to do what is obviously, trivially  safe 
with dip1000. If I can't do that, then there is something wrong with it.

-Steve
Oct 23 2018
parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Tuesday, 23 October 2018 at 17:48:23 UTC, Steven Schveighoffer 
wrote:


 Except that there should be no pointers to scope data here -- 
 the only thing that is scope really is the string[] pointer. 
 When you extract a string from that, it shouldn't be scope.

 If it is being inferred scope, how does one have a scope array 
 of non-scope pointers?


I *think* there's a bug with the implementation of DIP1000:

https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md#aggregates

- Lifetimes of built-in dynamically-sized slices T[] are analyzed 
as structs with two fields, one of type T* and the other of type 
size_t.

- For struct members of aggregate type, decomposition may 
continue transitively.

So, if "decomposition may continue transitively", it looks as if 
there's no tail-scope for arrays, but then shouldn't it follow 
that for

void foo(scope string[] args...);

it should transitively slap `scope` into the immutable(char)* of 
each argument?

However,

string[] escape;

// compiles, i.e. the behavior you essentially want
void foo(scope string[] args...) {
     foreach (s; args) escape ~= s;
}

// doesn't compile, i.e. the behavior you essentially get
void bar(scope string arg) {
     escape ~= s;
}

...and yet, in your code example the compiler behaves as if each 
string was indeed `scope`.
Oct 23 2018
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/23/18 2:43 PM, Stanislav Blinov wrote:

 On Tuesday, 23 October 2018 at 17:48:23 UTC, Steven Schveighoffer wrote:
 

 Except that there should be no pointers to scope data here -- the only 
 thing that is scope really is the string[] pointer. When you extract a 
 string from that, it shouldn't be scope.

 If it is being inferred scope, how does one have a scope array of 
 non-scope pointers?


 
 I *think* there's a bug with the implementation of DIP1000:
 
 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md#aggregates
 
 - Lifetimes of built-in dynamically-sized slices T[] are analyzed as 
 structs with two fields, one of type T* and the other of type size_t.
 
 - For struct members of aggregate type, decomposition may continue 
 transitively.
 
 So, if "decomposition may continue transitively", it looks as if there's 
 no tail-scope for arrays, but then shouldn't it follow that for
 
 void foo(scope string[] args...);
 
 it should transitively slap `scope` into the immutable(char)* of each 
 argument?


This is where I think there is going to be a problem. scope shouldn't be 
transitive, you can easily have a pointer allocated on the stack point 
to heap data. What is the type that results from taking the address of 
the pointer? It *should* be a pointer to a scoped pointer that points to 
NON-SCOPE data.

It may be an issue of expressability.


 
 However,
 
 string[] escape;
 
 // compiles, i.e. the behavior you essentially want
 void foo(scope string[] args...) {
      foreach (s; args) escape ~= s;
 }
 
 // doesn't compile, i.e. the behavior you essentially get
 void bar(scope string arg) {
      escape ~= s;
 }
 
 ....and yet, in your code example the compiler behaves as if each string 
 was indeed `scope`.


Yes, I believe the compiler inferring scope can do some things that 
syntax cannot. I think that might be where the issue is, but it's really 
hard to figure out.

-Steve
Oct 23 2018
parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Tuesday, 23 October 2018 at 19:03:25 UTC, Steven Schveighoffer 
wrote:


 So, if "decomposition may continue transitively", it looks as 
 if there's no tail-scope for arrays, but then shouldn't it 
 follow that for
 
 void foo(scope string[] args...);
 
 it should transitively slap `scope` into the immutable(char)* 
 of each argument?


 This is where I think there is going to be a problem. scope 
 shouldn't be transitive, you can easily have a pointer 
 allocated on the stack point to heap data.


Yeah, but OTOH, so can you have an array of pointers to the stack 
:\


 It may be an issue of expressability.


void tailScope(T scope [] x); // yuck...


 
 ....and yet, in your code example the compiler behaves as if 
 each string was indeed `scope`.


 Yes, I believe the compiler inferring scope can do some things 
 that syntax cannot. I think that might be where the issue is, 
 but it's really hard to figure out.


This is weird. 'foreach' compiles, manual iteration doesn't. 
Change the 'manualIter' enum below to compare.

/* Simplified versions of range primitives, for easy reference.
    Note, these don't handle decoding narrow strings!
*/

 property bool empty(T)(T[] a)  safe pure nothrow  nogc {
     return a.length == 0;
}

void popFront(T)(ref T[] a)  safe pure nothrow  nogc {
     assert(!a.empty);
     a = a[1 .. $];
}

/* version in std.range doesn't have a 'return' argument, but 
that doesn't seem to have an effect. */
 property ref T front(T)(return T[] a)  safe pure nothrow  nogc {
     assert(a.length);
     return a[0];
}

struct Array(T) {
     T[] data;

     void append(Stuff)(Stuff stuff) {
         appendImpl(stuff);
     }

     void appendImpl(T[] x...) {
         appendImpl2(x);
     }

     void appendImpl2(Stuff)(ref scope Stuff stuff) {

         // Change this to true to get different behavior
         enum manualIter = false;

         static if (manualIter) {
             while (!stuff.empty) {
                 data ~= stuff.front;
                 stuff.popFront;
             }
         } else {
             foreach (ref e; stuff) data ~= e;
         }
     }
}

void main() {
     Array!string array;
     array.append("hello");
}
Oct 23 2018
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/23/18 4:10 PM, Stanislav Blinov wrote:

 On Tuesday, 23 October 2018 at 19:03:25 UTC, Steven Schveighoffer wrote:
 

 So, if "decomposition may continue transitively", it looks as if 
 there's no tail-scope for arrays, but then shouldn't it follow that for

 void foo(scope string[] args...);

 it should transitively slap `scope` into the immutable(char)* of each 
 argument?


 This is where I think there is going to be a problem. scope shouldn't 
 be transitive, you can easily have a pointer allocated on the stack 
 point to heap data.


 
 Yeah, but OTOH, so can you have an array of pointers to the stack :\


This is where dip1000 actually makes sense -- the lifetime of that array 
should not outlive the stack. So, with dip1000 you *shouldn't* be able 
to have a heap array of pointers to the stack. Unless you circumvent via 
some specially written type.

-Steve
Oct 23 2018
parent Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Tuesday, 23 October 2018 at 20:46:26 UTC, Steven Schveighoffer 
wrote:


 This is where I think there is going to be a problem. scope 
 shouldn't be transitive, you can easily have a pointer 
 allocated on the stack point to heap data.


 
 Yeah, but OTOH, so can you have an array of pointers to the 
 stack :\


 This is where dip1000 actually makes sense -- the lifetime of 
 that array should not outlive the stack. So, with dip1000 you 
 *shouldn't* be able to have a heap array of pointers to the 
 stack.


Indeed, but that's for heap arrays. This will (and should) work:

void sink(scope immutable(char)*[] arr)  safe {}
void test()  safe { // even though putting  nogc here will still 
result in 'may cause GC allocation'...
     immutable char c;
     sink([&c]); // ...this is actually fine
}

Now for the meat. It looks like it just won't let you  safe-ly 
store away `scope` strings in that List:

string[] hatch;

void escape(string s)  safe { hatch ~= s; }

// templated to force attribute inference
void testInferred()(scope string[] stuff) {
     static string[] hatch;

     while (!stuff.empty) {

         // this blatantly and silently strips  safe from 
`testInferred`
         // without compile errors
         escape(stuff.front);

         // however, this will result in compilation error:
         //hatch ~= stuff.front;

         stuff.popFront;
     }
}

// fails to compile outright
void testSafe(scope string[] stuff)  safe {

     while (!stuff.empty) {

         // in  safe scope can't escape like this at all
         escape(stuff.front);

         stuff.popFront;
     }
}

void main() {
     testInferred(["hello"]); // compiles

     // fails to compile, testInferred is  system!
     ()  safe {
         testInferred(["hello"]);
     } ();
}

...so transitivity strikes yet again, albeit from the shadows 
this time.
Oct 23 2018
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/23/2018 8:10 AM, Steven Schveighoffer wrote:

 So, here is one other thing I want to say. This took me HOURS to find, and 
 narrow down. Not because I don't understand the concepts behind dip1000, but 
 because the compiler has fully inserted so many hidden scopes, I don't know
what 
 the actual code it's compiling is. One big problem I think with dip1000 is 
 simply that it's nearly impossible to understand where the issues are. Like I 
 said at the end of the post above, the result of allowing compiler inference
of 
 dip1000 is that your whole program is simply marked unsafe, and you have 
 absolutely no idea where it is. You can't even guess, because scope just shows 
 up where you never typed it. Given that you NEED this functionality on 
 templates, it's going to result, IMO, in people either not using dip1000, or 
 giving up and adding  trusted: to the top of their file. This is going to be 
 horrible if we can't find a way to either require scope instead of inferring
it 
 in some cases, or create a way to diagnose where the blasted problem actually 
 is. Maybe something to say "I expected this call to be  safe, why isn't it".


My improvements to DIP1000 are completely dead in the water due to lack of 
interest. It's impossible to make Phobos DIP1000 compatible if nobody is
willing 
to approve the improvements.

https://github.com/dlang/dmd/pull/8504
Oct 23 2018
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 24/10/2018 11:10 AM, Walter Bright wrote:

 On 10/23/2018 8:10 AM, Steven Schveighoffer wrote:

 So, here is one other thing I want to say. This took me HOURS to find, 
 and narrow down. Not because I don't understand the concepts behind 
 dip1000, but because the compiler has fully inserted so many hidden 
 scopes, I don't know what the actual code it's compiling is. One big 
 problem I think with dip1000 is simply that it's nearly impossible to 
 understand where the issues are. Like I said at the end of the post 
 above, the result of allowing compiler inference of dip1000 is that 
 your whole program is simply marked unsafe, and you have absolutely no 
 idea where it is. You can't even guess, because scope just shows up 
 where you never typed it. Given that you NEED this functionality on 
 templates, it's going to result, IMO, in people either not using 
 dip1000, or giving up and adding  trusted: to the top of their file. 
 This is going to be horrible if we can't find a way to either require 
 scope instead of inferring it in some cases, or create a way to 
 diagnose where the blasted problem actually is. Maybe something to say 
 "I expected this call to be  safe, why isn't it".


 
 My improvements to DIP1000 are completely dead in the water due to lack 
 of interest. It's impossible to make Phobos DIP1000 compatible if nobody 
 is willing to approve the improvements.
 
 https://github.com/dlang/dmd/pull/8504


Did the spec get the update that was requested over DIP1000?
Oct 23 2018
parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Tuesday, 23 October 2018 at 22:12:30 UTC, rikki cattermole 
wrote:

 On 24/10/2018 11:10 AM, Walter Bright wrote:

 My improvements to DIP1000 are completely dead in the water 
 due to lack of interest. It's impossible to make Phobos 
 DIP1000 compatible if nobody is willing to approve the 
 improvements.
 
 https://github.com/dlang/dmd/pull/8504


 Did the spec get the update that was requested over DIP1000?


No, _I did the work for it_ 
(https://github.com/dlang/dlang.org/pull/2453) but Walter has 
simply chosen to not respond, _at all_.

I can't merge those changes, 1) I don't have merge rights, 2) 
even if I did it, requires a review, and 3) the only person who 
can reasonably review it is Walter, who is not responding.
Oct 23 2018
prev sibling next sibling parent reply Brad Roberts <braddr puremagic.com> writes:
On 10/23/2018 3:10 PM, Walter Bright via Digitalmars-d wrote:

 
 My improvements to DIP1000 are completely dead in the water due to lack 
 of interest. It's impossible to make Phobos DIP1000 compatible if nobody 
 is willing to approve the improvements.
 
 https://github.com/dlang/dmd/pull/8504


Every time you say this I want to drive over and bonk you up side the 
head.  You keep making this statement and it's almost entirely false. 
The onus has been on you to produce specs and docs for this major change 
to the language semantics.  You inevitably point to the one issue in 
bugzilla, which is then pointed out to be sketchy and incomplete and the 
topic fades off into history, again.

If you want this to make forward progress, and I think it's fairly 
agreed that it needs to, you really need to accept that it's past time 
to do the documentation work.

Sigh,
Brad
Oct 23 2018
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Tuesday, 23 October 2018 at 22:22:18 UTC, Brad Roberts wrote:

 On 10/23/2018 3:10 PM, Walter Bright via Digitalmars-d wrote:

 
 My improvements to DIP1000 are completely dead in the water 
 due to lack of interest. It's impossible to make Phobos 
 DIP1000 compatible if nobody is willing to approve the 
 improvements.
 
 https://github.com/dlang/dmd/pull/8504


 Every time you say this I want to drive over and bonk you up 
 side the head.


Please do.


 You keep making this statement and it's almost entirely false. 
 The onus has been on you to produce specs and docs for this 
 major change to the language semantics.  You inevitably point 
 to the one issue in bugzilla, which is then pointed out to be 
 sketchy and incomplete


and not the place for documentation or specification.


 and the topic fades off into history, again.


And he wonders why!


 If you want this to make forward progress, and I think it's 
 fairly agreed that it needs to, you really need to accept that 
 it's past time to do the documentation work.


He doesn't need to, I did it for him: 
https://github.com/dlang/dmd/pull/8504
He just needs to review it.


 Sigh,


True dat.
Oct 23 2018
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/23/2018 4:56 PM, Nicholas Wilson wrote:

 He doesn't need to, I did it for him: https://github.com/dlang/dmd/pull/8504
 He just needs to review it.


https://github.com/dlang/dlang.org/pull/2453

While I thank and appreciate you for doing this work, and especially for taking 
the initiative instead of just complaining, I don't think that modifying a DIP 
that has already been approved is appropriate process. It should be a PR
against 
the spec. Approved specs should be immutable.

In the meantime, an Enhancement Request in Bugzilla is perfectly adequate 
documentation for a minor amendment, which is what this is. It can be merged 
into the spec later. We do this all the time.
Oct 23 2018
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 24 October 2018 at 01:31:33 UTC, Walter Bright 
wrote:

 On 10/23/2018 4:56 PM, Nicholas Wilson wrote:

 He doesn't need to, I did it for him: 
 https://github.com/dlang/dmd/pull/8504
 He just needs to review it.


 https://github.com/dlang/dlang.org/pull/2453

 While I thank and appreciate you for doing this work, and 
 especially for taking the initiative instead of just 
 complaining, I don't think that modifying a DIP that has 
 already been approved is appropriate process.


Yes, but you did it anyway (and I know that not what you meant it 
to mean).


 It should be a PR against the spec. Approved specs should be 
 immutable.


Be that as it may, documentation that does not reflect the status 
quo is worse than useless.


 In the meantime, an Enhancement Request in Bugzilla is 
 perfectly adequate documentation for a minor amendment, which 
 is what this is. It can be merged into the spec later. We do 
 this all the time.


You know that opinion is not held by a number of people, most 
notably those reviewing your PRs. Because we do not understand 
the changes, because they are not documented we are not qualified 
to review. And if nobody is able to review your PRs because, you, 
through your actions, have reduced that set to zero, then you 
have only yourself to blame.
Oct 23 2018
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/23/2018 7:38 PM, Nicholas Wilson wrote:

 Because we do not understand the changes,


Ask what it is you do not understand.
Oct 24 2018
parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 24 October 2018 at 20:36:03 UTC, Walter Bright 
wrote:

 On 10/23/2018 7:38 PM, Nicholas Wilson wrote:

 Because we do not understand the changes,


 Ask what it is you do not understand.
Oct 25 2018
prev sibling next sibling parent reply Neia Neutuladh <neia ikeran.org> writes:
On Tue, 23 Oct 2018 18:31:33 -0700, Walter Bright wrote:


 On 10/23/2018 4:56 PM, Nicholas Wilson wrote:

 He doesn't need to, I did it for him:
 https://github.com/dlang/dmd/pull/8504 He just needs to review it.


 
 https://github.com/dlang/dlang.org/pull/2453
 
 While I thank and appreciate you for doing this work, and especially for
 taking the initiative instead of just complaining, I don't think that
 modifying a DIP that has already been approved is appropriate process.
 It should be a PR against the spec. Approved specs should be immutable.
 
 In the meantime, an Enhancement Request in Bugzilla is perfectly
 adequate documentation for a minor amendment, which is what this is. It
 can be merged into the spec later. We do this all the time.


The change seems small to you, so documenting it at the same time as the 
PR is out doesn't seem that important. But it still needs to be 
documented, and people are asking for the documentation to help with 
reviewing the PR. Why not just write the documentation? It's been three 
months.
Oct 23 2018
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/23/2018 10:20 PM, Neia Neutuladh wrote:

 The change seems small to you, so documenting it at the same time as the
 PR is out doesn't seem that important. But it still needs to be
 documented, and people are asking for the documentation to help with
 reviewing the PR. Why not just write the documentation? It's been three
 months.



It is documented here: https://issues.dlang.org/show_bug.cgi?id=19097

If you don't understand it, feel free to ask.
Oct 24 2018
next sibling parent reply 12345swordy <alexanderheistermann gmail.com> writes:
On Wednesday, 24 October 2018 at 20:51:11 UTC, Walter Bright 
wrote:

 On 10/23/2018 10:20 PM, Neia Neutuladh wrote:

 The change seems small to you, so documenting it at the same 
 time as the
 PR is out doesn't seem that important. But it still needs to be
 documented, and people are asking for the documentation to 
 help with
 reviewing the PR. Why not just write the documentation? It's 
 been three
 months.



 It is documented here: 
 https://issues.dlang.org/show_bug.cgi?id=19097

 If you don't understand it, feel free to ask.


Bugzilla is meant to report bugs, not document features. You risk 
alienating the rookies here.

-Alex
Oct 24 2018
next sibling parent welkam <wwwelkam gmail.com> writes:
On Wednesday, 24 October 2018 at 21:01:17 UTC, 12345swordy wrote:

 Bugzilla is meant to report bugs, not document features. You 
 risk alienating the rookies here.

 -Alex


As a rookie I agree with Alex. I would not look for documentation 
in bugzilla. D language is big and compiler internals are 
complex. I need all documentation I can get
Oct 24 2018
prev sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 24 October 2018 at 21:01:17 UTC, 12345swordy wrote:

 On Wednesday, 24 October 2018 at 20:51:11 UTC, Walter Bright 
 wrote:

 On 10/23/2018 10:20 PM, Neia Neutuladh wrote:

 The change seems small to you, so documenting it at the same 
 time as the
 PR is out doesn't seem that important. But it still needs to 
 be
 documented, and people are asking for the documentation to 
 help with
 reviewing the PR. Why not just write the documentation? It's 
 been three
 months.



 It is documented here: 
 https://issues.dlang.org/show_bug.cgi?id=19097

 If you don't understand it, feel free to ask.


 Bugzilla is meant to report bugs, not document features.


We have told Walter that so many times before, I've lost count 
and he still doesn't listen. I don't know why I bother.


 You risk alienating the rookies here.


That statement is wrong for two reasons:
1) there is no risk, it has happened, and
2) it applies to everyone, not just the rookies. Atila, who has 
done more production use of dip1000 than anyone else I know, was 
recently (for some value of recently) surprised at the way it 
worked in some way (that I can't be bothered to cite at the 
moment, jet lag).
Oct 25 2018
prev sibling next sibling parent reply Neia Neutuladh <neia ikeran.org> writes:
On Wed, 24 Oct 2018 13:51:11 -0700, Walter Bright wrote:

 On 10/23/2018 10:20 PM, Neia Neutuladh wrote:

 The change seems small to you, so documenting it at the same time as
 the PR is out doesn't seem that important. But it still needs to be
 documented, and people are asking for the documentation to help with
 reviewing the PR. Why not just write the documentation? It's been three
 months.


 
 
 It is documented here: https://issues.dlang.org/show_bug.cgi?id=19097
 
 If you don't understand it, feel free to ask.


People are asking for a spec update with specification-level clarity and 
precision. This will be required prior to the next major release after 
that PR gets submitted. Why not do it now?
Oct 24 2018
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 24 October 2018 at 22:26:48 UTC, Neia Neutuladh 
wrote:

 On Wed, 24 Oct 2018 13:51:11 -0700, Walter Bright wrote:

 It is documented here: 
 https://issues.dlang.org/show_bug.cgi?id=19097
 
 If you don't understand it, feel free to ask.


 People are asking for a spec update with specification-level 
 clarity and precision.


We're not even asking for that! 
github.com/dlang/dlang.org/pull/2453 is mostly the "how do you 
use this feature, and why is that important", not the nitty 
gritty. Without the "how to use" and "why" it is impossible to 
differentiate the wrong nitty gritty from the right nitty gritty 
(which is what reviewing is).
Oct 25 2018
parent reply Neia Neutuladh <neia ikeran.org> writes:
On Thu, 25 Oct 2018 07:36:48 +0000, Nicholas Wilson wrote:

 We're not even asking for that! github.com/dlang/dlang.org/pull/2453 is
 mostly the "how do you use this feature, and why is that important", not
 the nitty gritty. Without the "how to use" and "why" it is impossible to
 differentiate the wrong nitty gritty from the right nitty gritty (which
 is what reviewing is).


Huh, I read the bugzilla entry and it seemed moderately clear.
Oct 25 2018
next sibling parent 12345swordy <alexanderheistermann gmail.com> writes:
On Thursday, 25 October 2018 at 15:57:14 UTC, Neia Neutuladh 
wrote:

 On Thu, 25 Oct 2018 07:36:48 +0000, Nicholas Wilson wrote:

 We're not even asking for that! 
 github.com/dlang/dlang.org/pull/2453 is mostly the "how do you 
 use this feature, and why is that important", not the nitty 
 gritty. Without the "how to use" and "why" it is impossible to 
 differentiate the wrong nitty gritty from the right nitty 
 gritty (which is what reviewing is).


 Huh, I read the bugzilla entry and it seemed moderately clear.


That not the point. People will first look at the spec for 
feature usage, not Bugzilla.
Oct 25 2018
prev sibling parent reply sclytrack <fake hotmail.com> writes:
On Thursday, 25 October 2018 at 15:57:14 UTC, Neia Neutuladh 
wrote:

 On Thu, 25 Oct 2018 07:36:48 +0000, Nicholas Wilson wrote:

 We're not even asking for that! 
 github.com/dlang/dlang.org/pull/2453 is mostly the "how do you 
 use this feature, and why is that important", not the nitty 
 gritty. Without the "how to use" and "why" it is impossible to 
 differentiate the wrong nitty gritty from the right nitty 
 gritty (which is what reviewing is).


 Huh, I read the bugzilla entry and it seemed moderately clear.



Lolz, one person wants the spec first and the other one wants the 
code first.

spec: https://github.com/dlang/dlang.org/pull/2453
code: https://github.com/dlang/dmd/pull/8504

Both changes to the spec and changes to the code have been 
written.
Everybody is holding their ground, never surrender.
It's been going on for 2 months. Hilarious.
Oct 25 2018
parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Thursday, 25 October 2018 at 21:07:39 UTC, sclytrack wrote:

 On Thursday, 25 October 2018 at 15:57:14 UTC, Neia Neutuladh 
 wrote:

 On Thu, 25 Oct 2018 07:36:48 +0000, Nicholas Wilson wrote:

 We're not even asking for that! 
 github.com/dlang/dlang.org/pull/2453 is mostly the "how do 
 you use this feature, and why is that important", not the 
 nitty gritty. Without the "how to use" and "why" it is 
 impossible to differentiate the wrong nitty gritty from the 
 right nitty gritty (which is what reviewing is).


 Huh, I read the bugzilla entry and it seemed moderately clear.



 Lolz, one person wants the spec first and the other one wants 
 the code first.

 spec: https://github.com/dlang/dlang.org/pull/2453
 code: https://github.com/dlang/dmd/pull/8504

 Both changes to the spec and changes to the code have been 
 written.
 Everybody is holding their ground, never surrender.
 It's been going on for 2 months. Hilarious.


Unfortunately, its not quite that funny: I don't know that the 
spec PR accurately reflects the code because I didn't write the 
code. I also don't have the authority to merge either of those 
PRs, hence the "deadlock" is Walter's to break.
Oct 25 2018
prev sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 24 October 2018 at 20:51:11 UTC, Walter Bright 
wrote:

 On 10/23/2018 10:20 PM, Neia Neutuladh wrote:

 The change seems small to you, so documenting it at the same 
 time as the
 PR is out doesn't seem that important. But it still needs to be
 documented, and people are asking for the documentation to 
 help with
 reviewing the PR. Why not just write the documentation? It's 
 been three
 months.



 It is documented here: 
 https://issues.dlang.org/show_bug.cgi?id=19097


For the last time: BUGZILLA IS NOT DOCUMENTATION!!!!!!!!!


 If you don't understand it, feel free to ask.


I have: 
https://github.com/dlang/dlang.org/pull/2453#issue-210353863
Oct 25 2018
prev sibling parent reply Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Wednesday, 24 October 2018 at 01:31:33 UTC, Walter Bright 
wrote:

 On 10/23/2018 4:56 PM, Nicholas Wilson wrote:

 He doesn't need to, I did it for him: 
 https://github.com/dlang/dmd/pull/8504
 He just needs to review it.


 https://github.com/dlang/dlang.org/pull/2453

 While I thank and appreciate you for doing this work, and 
 especially for taking the initiative instead of just 
 complaining, I don't think that modifying a DIP that has 
 already been approved is appropriate process. It should be a PR 
 against the spec. Approved specs should be immutable.


[...]

With all due respect, why wasn't this brought forward as a 
comment on the PR back in August? (And it _is_ a PR against the 
spec, is it not?) I feel embarrassed for the image that is being 
presented outwards, which is one of central people being 
incapable of working together. I don't understand what is going 
on; how could the language evolve this far and what has changed? 
Do we have too much regulation and procedures or too little? 
Isn't Walter's explanation in bugzilla enough this time to review 
his PR and keep the language improving while procedures are being 
improved on in parallel? I feel like people are keeping a stiff 
foot for far too long. Stagnation in development is bad enough, 
but this image really hurts me. I count on you, I want you guys 
to shine. I see all these talented musicians, but the conductor 
is not on his podium.

Come on, friends, you've cracked harder nuts. Please :-)

Bastiaan.
Oct 24 2018
parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Thursday, 25 October 2018 at 00:20:17 UTC, Bastiaan Veelo 
wrote:

 On Wednesday, 24 October 2018 at 01:31:33 UTC, Walter Bright 
 wrote:

 On 10/23/2018 4:56 PM, Nicholas Wilson wrote:

 He doesn't need to, I did it for him: 
 https://github.com/dlang/dmd/pull/8504
 He just needs to review it.


 https://github.com/dlang/dlang.org/pull/2453

 While I thank and appreciate you for doing this work, and 
 especially for taking the initiative instead of just 
 complaining, I don't think that modifying a DIP that has 
 already been approved is appropriate process. It should be a 
 PR against the spec. Approved specs should be immutable.


 [...]

 With all due respect,


I'm pretty sure all respect is past its dues on this, but 
anyway...


 why wasn't this brought forward as a comment on the PR back in 
 August?


I don't know, Walter hasn't bothered to reply.


 (And it _is_ a PR against the spec, is it not?)


Technically yes. But what should we document, the spec or the 
status quo?


 I feel embarrassed for the image that is being presented 
 outwards, which is one of central people being incapable of 
 working together. I don't understand what is going on; how 
 could the language evolve this far and what has changed? Do we 
 have too much regulation and procedures or too little?



 Isn't Walter's explanation in bugzilla enough this time to 
 review his PR and keep the language improving while procedures 
 are being improved on in parallel?



that the documentations changes would follow (this is indeed 
sometimes a useful pattern so we went through with it).


review it, on principle and in practice. The principle sets a bad 
example which I can only guess that Walter will keep doing. The 
practical is that if we do not understand the PR there is no way 
we can review it.


 I feel like people are keeping a stiff foot for far too long.


Walter is the critical point on the critical path at the moment.


 Stagnation in development is bad enough, but this image really 
 hurts me. I count on you, I want you guys to shine. I see all 
 these talented musicians, but the conductor is not on his 
 podium.


That is a very apt analogy, you should talk to the conductor. I 
hope to announce the answer i have for this soon, but it will be 
by necessity a high latency process.


 Come on, friends, you've cracked harder nuts. Please :-)


Indeed, the next move is Walter's, if he chooses to not make it, 
so be it.
Oct 25 2018
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/23/2018 3:22 PM, Brad Roberts wrote:

 On 10/23/2018 3:10 PM, Walter Bright via Digitalmars-d wrote:

 My improvements to DIP1000 are completely dead in the water due to lack of 
 interest. It's impossible to make Phobos DIP1000 compatible if nobody is 
 willing to approve the improvements.

 https://github.com/dlang/dmd/pull/8504


 
 Every time you say this I want to drive over and bonk you up side the head. 
You 
 keep making this statement and it's almost entirely false. The onus has been
on 
 you to produce specs and docs for this major change to the language semantics.
 
 You inevitably point to the one issue in bugzilla, which is then pointed out
to 
 be sketchy and incomplete and the topic fades off into history, again.


No, it has not been pointed out so. There was one request for clarification, 
which I responded to. https://issues.dlang.org/show_bug.cgi?id=19097#c5

There's not been a single comment on the PR implementation itself.



 If you want this to make forward progress, and I think it's fairly agreed that 
 it needs to, you really need to accept that it's past time to do the 
 documentation work.


The documentation is there for all to read.

There's no consistent practice for the documentation must be pulled before the 
implementation or vice versa.
Oct 23 2018
parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 24 October 2018 at 01:23:00 UTC, Walter Bright 
wrote:

 On 10/23/2018 3:22 PM, Brad Roberts wrote:

 On 10/23/2018 3:10 PM, Walter Bright via Digitalmars-d wrote:

 My improvements to DIP1000 are completely dead in the water 
 due to lack of interest. It's impossible to make Phobos 
 DIP1000 compatible if nobody is willing to approve the 
 improvements.

 https://github.com/dlang/dmd/pull/8504


 
 Every time you say this I want to drive over and bonk you up 
 side the head.  You keep making this statement and it's almost 
 entirely false. The onus has been on you to produce specs and 
 docs for this major change to the language semantics.  You 
 inevitably point to the one issue in bugzilla, which is then 
 pointed out to be sketchy and incomplete and the topic fades 
 off into history, again.


 No, it has not been pointed out so. There was one request for 
 clarification, which I responded to. 
 https://issues.dlang.org/show_bug.cgi?id=19097#c5

 There's not been a single comment on the PR implementation 
 itself.


Yes, can you guess why?


 If you want this to make forward progress, and I think it's 
 fairly agreed that it needs to, you really need to accept that 
 it's past time to do the documentation work.


 The documentation is there for all to read.


FFS! Bugzilla in NOT documentation!


 There's no consistent practice for the documentation must be 
 pulled before the implementation or vice versa.


You have _three_ PRs outstanding lacking documentation, which _I_
have documented _for you_, all you need do is verify that what I 
think you have have done in those PRs and we can move forward.

Until you do that nobody is going to review your PRs.

You're already caused at least two people to "reconsider their 
involvement" over the way you have handled dip1008, if you 
continue your arrogance/ignorance/stubbornness (and I'm genuinely 
not sure which) that number is only going to rise.

Shachar was right, D does have lethal structural problems just 
waiting to start crumbling.

I even agree with the change you want to make, but I don't 
understand it because there's no fucking documentation!

/rant
Oct 23 2018
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/23/18 6:10 PM, Walter Bright wrote:

 On 10/23/2018 8:10 AM, Steven Schveighoffer wrote:

 So, here is one other thing I want to say. This took me HOURS to find, 
 and narrow down. Not because I don't understand the concepts behind 
 dip1000, but because the compiler has fully inserted so many hidden 
 scopes, I don't know what the actual code it's compiling is. One big 
 problem I think with dip1000 is simply that it's nearly impossible to 
 understand where the issues are. Like I said at the end of the post 
 above, the result of allowing compiler inference of dip1000 is that 
 your whole program is simply marked unsafe, and you have absolutely no 
 idea where it is. You can't even guess, because scope just shows up 
 where you never typed it. Given that you NEED this functionality on 
 templates, it's going to result, IMO, in people either not using 
 dip1000, or giving up and adding  trusted: to the top of their file. 
 This is going to be horrible if we can't find a way to either require 
 scope instead of inferring it in some cases, or create a way to 
 diagnose where the blasted problem actually is. Maybe something to say 
 "I expected this call to be  safe, why isn't it".


 
 My improvements to DIP1000 are completely dead in the water due to lack 
 of interest. It's impossible to make Phobos DIP1000 compatible if nobody 
 is willing to approve the improvements.
 
 https://github.com/dlang/dmd/pull/8504


I know this debate is continuing. But that PR has nothing to do with the 
problem I'm having (what this post is really about). I'm not returning 
anything via the first parameter. Just for kicks, I downloaded your PR 
branch, and it still has the same problem with somehow assuming 
something is scope. So can you address this problem at all?

Alarmingly, the current dlist builds fine with dip1000, even though it 
conceptually is the same thing -- it's just that the casts destroy any 
tracking of lifetimes (or so my theory goes), so it builds happily. We 
don't want people to resort to opacity and casts to get around dip1000.

-Steve
Oct 25 2018
next sibling parent Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Thursday, 25 October 2018 at 13:50:15 UTC, Steven 
Schveighoffer wrote:


 Alarmingly, the current dlist builds fine with dip1000, even 
 though it conceptually is the same thing -- it's just that the 
 casts destroy any tracking of lifetimes (or so my theory goes), 
 so it builds happily. We don't want people to resort to opacity 
 and casts to get around dip1000.


It's the transitivity of `scope` coupled with that other line in 
the spec that just disables errors in non- safe code.

Lowering your code, conceptually what's happening is that you're 
defining this:

struct Storage(T) {
     void add(scope T x);
}

which means you want to escape 'x'.

Now, I understand that that is not the *intent* of your code, but 
that's what the compiler sees. I'd argue that the "correct" thing 
to do here is leave the range insertion to some OutputRange 
algorithm. Unless we can find a way to define tail-`scope` arrays.
Oct 25 2018
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/25/2018 6:50 AM, Steven Schveighoffer wrote:

 I know this debate is continuing. But that PR has nothing to do with the
problem 
 I'm having (what this post is really about). I'm not returning anything via
the 
 first parameter. Just for kicks, I downloaded your PR branch, and it still has 
 the same problem with somehow assuming something is scope. So can you address 
 this problem at all?


The problems people have with DIP1000 nearly always stem from the code being 
complicated. The code example you posted is quite complicated with a lot of 
moving parts. The only way to get a handle on what is happening is to break it 
down into small, and I mean small, chunks.

The trick to finding out why a chunk of code is being inferred as not  safe is 
to mark it  safe and thereby see where the error is.

Some designs are unworkable with scope without some unsafe casting. A linked 
list may be one of them (I believe the same holds for linked lists in Rust, but 
I'm not sure if I remember that correctly).

And lastly, I've done this with Phobos. Phobos is absurdly and ridiculously 
tangled code with layers and layers and layers of templates that all just 
forward to yet another template. It makes it pretty hard to track down DIP1000 
errors. DIP1000 is not terribly usable when Phobos can't be built with DIP1000, 
and that requires the PR to make progress on that.

Therefore DIP1000 is pretty dead in the water without the PR.
Oct 25 2018
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/25/18 4:37 PM, Walter Bright wrote:

 On 10/25/2018 6:50 AM, Steven Schveighoffer wrote:

 I know this debate is continuing. But that PR has nothing to do with 
 the problem I'm having (what this post is really about). I'm not 
 returning anything via the first parameter. Just for kicks, I 
 downloaded your PR branch, and it still has the same problem with 
 somehow assuming something is scope. So can you address this problem 
 at all?


 
 The problems people have with DIP1000 nearly always stem from the code 
 being complicated. The code example you posted is quite complicated with 
 a lot of moving parts. The only way to get a handle on what is happening 
 is to break it down into small, and I mean small, chunks.
 
 The trick to finding out why a chunk of code is being inferred as not 
  safe is to mark it  safe and thereby see where the error is.


I did do that. If you read my story above the rant, I added  safe until 
I got down to the error.

What I can't figure out is why I'm passing a string to the allocation 
function, which is allocating a new node on the heap containing the 
string, but it seems to want to say this should be scope. Both the 
string and the new node shouldn't be scope.


 Some designs are unworkable with scope without some unsafe casting. A 
 linked list may be one of them (I believe the same holds for linked 
 lists in Rust, but I'm not sure if I remember that correctly).


This sounds really disturbing. But I see things like this: 
https://rcoh.me/posts/rust-linked-list-basically-impossible/

And it does seem like maybe there simply isn't an easy way to do this. 
Maybe I do have to carefully figure out the casting and  trusted code, 
and just go ahead with the overrides. If Rust, with it's far more tested 
and tried borrow checker can't make this doable without escaping into 
unsafety, then probably dip1000 isn't going to be able to either.

Still something seems off when I have a type with infinite lifetime 
(string) and I want to put it into another type with infinite lifetime 
(Node) and it doesn't allow me to do it, because the strings came in on 
a scope array. I still feel there is a problem here that may be 
fundamental to the design of dip1000.


 And lastly, I've done this with Phobos. Phobos is absurdly and 
 ridiculously tangled code with layers and layers and layers of templates 
 that all just forward to yet another template. It makes it pretty hard 
 to track down DIP1000 errors. DIP1000 is not terribly usable when Phobos 
 can't be built with DIP1000, and that requires the PR to make progress 
 on that.
 
 Therefore DIP1000 is pretty dead in the water without the PR.


If rampant dip1000 cheating is happening like in dlist, then making 
phobos compile isn't the complete answer IMO. But you are right that 
until Phobos does compile with dip1000, you pretty much can't figure out 
anything that uses phobos.

I'm OK with the PR you linked, I just have no idea how to review it, as 
I'm not a DMD developer. The concept seems specific to Phobos, and a 
little kludgy, but if it moves things forward, I'd be OK with it. The 
only thing that seems really strange to me is that `return` is not the 
return.

-Steve
Oct 25 2018
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/25/2018 2:12 PM, Steven Schveighoffer wrote:

 What I can't figure out is why I'm passing a string to the allocation
function, 
 which is allocating a new node on the heap containing the string, but it seems 
 to want to say this should be scope. Both the string and the new node
shouldn't 
 be scope.


Can you boil this down to the minimum?



 If rampant dip1000 cheating is happening like in dlist, then making phobos 
 compile isn't the complete answer IMO.


My intent is for it to work without cheating.



 I'm OK with the PR you linked, I just have no idea how to review it, as I'm
not 
 a DMD developer.


One of my ongoing goals is to make DMD easier to understand. Unfortunately,
when 
it does get easy, people then feel comfortable adding complexity until it is no 
longer understandable. I suspect this is one of those immutable laws like the 
Peter Principle:

   https://en.wikipedia.org/wiki/Peter_principle

I.e. all software is doomed to be incomprehensible. I'll still struggle against 
that, though :-) Currently I've been doing some refactoring to replace Visitor 
traversals with switch statements. I find the latter much simpler to
understand, 
for the simple (!) reason that it has no dependency on an external thing like 
the Visitor base class.



 The concept seems specific to Phobos, and a little kludgy, but 
 if it moves things forward, I'd be OK with it. The only thing that seems
really 
 strange to me is that `return` is not the return.


I tried to make it as narrow as I could (restricted to the first argument) to 
see if that works well enough. If it does, great. If it doesn't, it can be 
expanded later, which is much easier than trying to narrow it.

The other thing about it is it is purely additive, so there's no risk of 
breaking existing code. I.e. it's low risk.
Oct 26 2018
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/26/18 6:17 AM, Walter Bright wrote:

 On 10/25/2018 2:12 PM, Steven Schveighoffer wrote:

 What I can't figure out is why I'm passing a string to the allocation 
 function, which is allocating a new node on the heap containing the 
 string, but it seems to want to say this should be scope. Both the 
 string and the new node shouldn't be scope.


 
 Can you boil this down to the minimum?


I did: https://run.dlang.io/is/6xDFnr


 I'm OK with the PR you linked, I just have no idea how to review it, 
 as I'm not a DMD developer.


 
 One of my ongoing goals is to make DMD easier to understand. 
 Unfortunately, when it does get easy, people then feel comfortable 
 adding complexity until it is no longer understandable. I suspect this 
 is one of those immutable laws like the Peter Principle:
 
    https://en.wikipedia.org/wiki/Peter_principle
 
 I.e. all software is doomed to be incomprehensible. I'll still struggle 
 against that, though :-) Currently I've been doing some refactoring to 
 replace Visitor traversals with switch statements. I find the latter 
 much simpler to understand, for the simple (!) reason that it has no 
 dependency on an external thing like the Visitor base class.


Part of my lack of understanding is my lack of effort to try and 
understand it. Until then, I really can't say what would make it better 
or not.

But when sitting next to knowledgable people at dconf this year, it was 
relatively straightforward to find where I had to change things for my 
first dmd PR. It's like driving around a strange neighborhood with a 
local who knows all the back roads.

I need to revisit your talk from 2016.

-Steve
Oct 26 2018
prev sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Thursday, 25 October 2018 at 20:37:55 UTC, Walter Bright wrote:

 The problems people have with DIP1000 nearly always stem from 
 the code being complicated. The code example you posted is 
 quite complicated with a lot of moving parts. The only way to 
 get a handle on what is happening is to break it down into 
 small, and I mean small, chunks.


Maybe there is scope for someone to write a tool that lowers all 
of the complicated features to simpler constructs, complementing 
existing features like outputting the assembly and AST?
Oct 26 2018
parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/26/2018 6:47 AM, jmh530 wrote:

 Maybe there is scope for someone to write a tool that lowers all of the 
 complicated features to simpler constructs, complementing existing features
like 
 outputting the assembly and AST?


The complexity problem comes from the number of different ways that a pointer 
can be represented:

1. pointer
2. ref
3. dynamic array
4. class
5. delegate
6. uplevel reference to local

especially when they are combined, like a ref to a pointer. Understanding how
to 
use scope requires a thorough understanding of these (often implicit) pointers. 
This is why I recommend designing something with a prototype using raw
pointers, 
get that to work, then add in the other reference types.

I don't see any way to have any sort of lifetime annotations without this 
understanding.
Oct 26 2018