www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DIP1000 finds stack corruption bug in std.socket

reply Walter Bright <newshound2 digitalmars.com> writes:
https://github.com/dlang/phobos/pull/6919

The function returns a pointer to a local variable. The pre-DIP1000 code 
circumvented the compiler error message by passing the address through another 
variable. DIP1000 tracks the lifetimes across assignments, exposing such 
shenanigans.

These sorts of bugs can be very hard to track down by their effects.
Mar 18 2019
next sibling parent reply Olivier FAURE <couteaubleu gmail.com> writes:
On Tuesday, 19 March 2019 at 04:25:30 UTC, Walter Bright wrote:

 https://github.com/dlang/phobos/pull/6919

 The function returns a pointer to a local variable. The 
 pre-DIP1000 code circumvented the compiler error message by 
 passing the address through another variable. DIP1000 tracks 
 the lifetimes across assignments, exposing such shenanigans.

 These sorts of bugs can be very hard to track down by their 
 effects.


I don't want to be aggressive, but this feels like you're saying 
"Look at how amazing DIP-1000 is! Isn't it great?" right after 
criticisms of DIP-1000 have started to surface again.

I get that you're very proud of DIP-1000, and that it's a lot 
better than having only DIP-25, but it still leaves the concept 
of scope in D with fundamental flaws, that make full  safe  nogc 
apps next to impossible.
Mar 19 2019
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/19/2019 4:29 AM, Olivier FAURE wrote:

 I don't want to be aggressive, but this feels like you're saying "Look at how 
 amazing DIP-1000 is! Isn't it great?" right after criticisms of DIP-1000 have 
 started to surface again.


More like right after it found the problem in std.socket, a stack corruption 
problem that's been there for who knows how long. It's exactly the kind of 
problem dip1000 is there for, and worth pointing out.



 I get that you're very proud of DIP-1000, and that it's a lot better than
having 
 only DIP-25, but it still leaves the concept of scope in D with fundamental 
 flaws, that make full  safe  nogc apps next to impossible.


It doesn't have to cover 100% of the program to offer a great deal of value.

It's difficult to retrofit these sorts of things into an existing, complex 
language, where a lot of usage has grown up around unsafe practices. Just 
imagine how hard it would be to add this to C++. We're in far better shape than 
that.

I've got dip1000 working with nearly all of Phobos now, which is a crucial 
foundational step.
Mar 19 2019
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 19 March 2019 at 19:22:43 UTC, Walter Bright wrote:

 

 I've got dip1000 working with nearly all of Phobos now, which 
 is a crucial foundational step.


I hadn't realized that you were that far along. That's great.
Mar 19 2019
parent walker <growup_wei qq.com> writes:
On Tuesday, 19 March 2019 at 19:30:53 UTC, jmh530 wrote:

 On Tuesday, 19 March 2019 at 19:22:43 UTC, Walter Bright wrote:

 

 I've got dip1000 working with nearly all of Phobos now, which 
 is a crucial foundational step.


 I hadn't realized that you were that far along. That's great.


Maybe it is about the dip1000.mak

https://github.com/dlang/phobos/blob/master/dip1000.mak
Mar 19 2019
prev sibling parent reply Meta <jared771 gmail.com> writes:
On Tuesday, 19 March 2019 at 04:25:30 UTC, Walter Bright wrote:

 https://github.com/dlang/phobos/pull/6919

 The function returns a pointer to a local variable. The 
 pre-DIP1000 code circumvented the compiler error message by 
 passing the address through another variable. DIP1000 tracks 
 the lifetimes across assignments, exposing such shenanigans.

 These sorts of bugs can be very hard to track down by their 
 effects.


I think we have a ways to go with DIP1000/DIP25, but after 
actually putting them to use I am cautiously optimistic. I was 
expecting it to be a nightmare to annotate my recent pet project, 
but it actually wasn't too bad and only took an evening going 
from 0 to about 85% annotated.

My main concern is that it doesn't seem to do a single thing if 
your functions aren't marked as  safe; from what I can tell, 
`scope`, `return scope`, etc. are entirely ignored without that 
 safe, which can be confusing when you're trying to figure out 
why something isn't working, only to realize you forgot  safe. 
This is also a big plus, of course, and I attribute the relative 
ease of transitioning my codebase to this fact. It definitely 
makes the burden of compliance for large code bases much easier.

Documentation is also a big concern, of course. That and some 
bugs (or perhaps just unsupported use cases - see my other 
thread) are two problems that I expect is just a matter of time 
and a larger number of people using DIP1000/DIP25, finding all 
the corner cases that need to be ironed out.
Mar 19 2019
parent reply Walter Bright <newshound2 digitalmars.com> writes:
As of today I finally got all of Phobos to compile with DIP1000. The remaining 
issue is that things sometimes get mangled differently with DIP1000 than 
without, leading to linking errors.

But it's working with all compiled with -dip1000.
Mar 20 2019
next sibling parent Andre Pany <andre s-e-a-p.de> writes:
On Wednesday, 20 March 2019 at 10:00:39 UTC, Walter Bright wrote:

 As of today I finally got all of Phobos to compile with 
 DIP1000. The remaining issue is that things sometimes get 
 mangled differently with DIP1000 than without, leading to 
 linking errors.

 But it's working with all compiled with -dip1000.


Sounds great. What is the plan regarding the managing issue?

Kind regards
Andre
Mar 20 2019
prev sibling next sibling parent Jonathan Marler <johnnymarler gmail.com> writes:
On Wednesday, 20 March 2019 at 10:00:39 UTC, Walter Bright wrote:

 As of today I finally got all of Phobos to compile with 
 DIP1000. The remaining issue is that things sometimes get 
 mangled differently with DIP1000 than without, leading to 
 linking errors.

 But it's working with all compiled with -dip1000.


I was working through this issue a year ago.  Created this thread 
to discuss potential solutions:

"D Library Breakage" 
https://forum.dlang.org/thread/nbdzcwlgzvdcxnecefdk forum.dlang.org

I'd like to see a way for D to be able to pull compilation 
settings from pre-compiled librarys.  Settings like what versions 
they were compiled with, what optional features they used (i.e. 
dip1000).  This way, the interpretation of the library code that 
gets imported always matches what's actually in the pre-compiled 
library.

This could be done by adding that information to the library file 
or creating a new file that can be deployed alongside the library 
with those pre-compiled settings.
Mar 20 2019
prev sibling parent reply bitwise <bitwise.pvt gmail.com> writes:
On Wednesday, 20 March 2019 at 10:00:39 UTC, Walter Bright wrote:

 As of today I finally got all of Phobos to compile with 
 DIP1000. The remaining issue is that things sometimes get 
 mangled differently with DIP1000 than without, leading to 
 linking errors.

 But it's working with all compiled with -dip1000.


Just out of curiosity - did that involve removing all instances 
of the 'in' type qualifier?

I guess I'm still holding out hope that it will be rebranded as 
the counterpart of 'out', as a by-ref, in-only, parameter type 
that accepts rvalues ;)

It's disappointing to hear the current plans for scope are so 
limited. I originally thought that scope would be a general type 
qualifier which restricted anything it was applied to from being 
copied out of it's current scope.

I was hoping to be able to apply scope to range objects returned 
from containers so they couldn't leave the scope of where they 
were retrieved. I also saw that Atila used scope on a postblit, 
which also seems like something that should keep the range from 
leaving the scope where it was retrieved. If I understand 
correctly though, neither of these uses is officially supported 
or planned right now - is that correct?

Thanks,
     Bit
Mar 23 2019
parent reply Meta <jared771 gmail.com> writes:
On Saturday, 23 March 2019 at 16:32:00 UTC, bitwise wrote:

 It's disappointing to hear the current plans for scope are so 
 limited. I originally thought that scope would be a general 
 type qualifier which restricted anything it was applied to from 
 being copied out of it's current scope.

 I was hoping to be able to apply scope to range objects 
 returned from containers so they couldn't leave the scope of 
 where they were retrieved. I also saw that Atila used scope on 
 a postblit, which also seems like something that should keep 
 the range from leaving the scope where it was retrieved. If I 
 understand correctly though, neither of these uses is 
 officially supported or planned right now - is that correct?

 Thanks,
     Bit


I also had this misconception initially because that's how scope 
was originally defined to work; hence this thread: 
https://forum.dlang.org/thread/zayyoiupftrvbummxabk forum.dlang.org
Mar 23 2019
next sibling parent bitwise <bitwise.pvt gmail.com> writes:
On Saturday, 23 March 2019 at 19:58:34 UTC, Meta wrote:

 On Saturday, 23 March 2019 at 16:32:00 UTC, bitwise wrote:

     [...]


 I also had this misconception initially because that's how 
 scope was originally defined to work; hence this thread: 
 https://forum.dlang.org/thread/zayyoiupftrvbummxabk forum.dlang.org


It makes no sense to have one keyword that overlaps two others, 
especially when there's such a strong bias against adding new 
keywords.
Mar 25 2019
prev sibling parent bitwise <bitwise.pvt gmail.com> writes:
On Saturday, 23 March 2019 at 19:58:34 UTC, Meta wrote:

 [...]


That was a serious question about "in".

(this point was supposed to precede the last comment, but was 
eaten by the forum)
Mar 25 2019