www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DIP1000 finds stack corruption bug in std.socket

reply Walter Bright <newshound2 digitalmars.com> writes:
https://github.com/dlang/phobos/pull/6919

The function returns a pointer to a local variable. The pre-DIP1000 code 
circumvented the compiler error message by passing the address through another 
variable. DIP1000 tracks the lifetimes across assignments, exposing such 
shenanigans.

These sorts of bugs can be very hard to track down by their effects.
Mar 18
next sibling parent reply Olivier FAURE <couteaubleu gmail.com> writes:
On Tuesday, 19 March 2019 at 04:25:30 UTC, Walter Bright wrote:
 https://github.com/dlang/phobos/pull/6919

 The function returns a pointer to a local variable. The 
 pre-DIP1000 code circumvented the compiler error message by 
 passing the address through another variable. DIP1000 tracks 
 the lifetimes across assignments, exposing such shenanigans.

 These sorts of bugs can be very hard to track down by their 
 effects.
I don't want to be aggressive, but this feels like you're saying "Look at how amazing DIP-1000 is! Isn't it great?" right after criticisms of DIP-1000 have started to surface again. I get that you're very proud of DIP-1000, and that it's a lot better than having only DIP-25, but it still leaves the concept of scope in D with fundamental flaws, that make full safe nogc apps next to impossible.
Mar 19
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/19/2019 4:29 AM, Olivier FAURE wrote:
 I don't want to be aggressive, but this feels like you're saying "Look at how 
 amazing DIP-1000 is! Isn't it great?" right after criticisms of DIP-1000 have 
 started to surface again.
More like right after it found the problem in std.socket, a stack corruption problem that's been there for who knows how long. It's exactly the kind of problem dip1000 is there for, and worth pointing out.
 I get that you're very proud of DIP-1000, and that it's a lot better than
having 
 only DIP-25, but it still leaves the concept of scope in D with fundamental 
 flaws, that make full  safe  nogc apps next to impossible.
It doesn't have to cover 100% of the program to offer a great deal of value. It's difficult to retrofit these sorts of things into an existing, complex language, where a lot of usage has grown up around unsafe practices. Just imagine how hard it would be to add this to C++. We're in far better shape than that. I've got dip1000 working with nearly all of Phobos now, which is a crucial foundational step.
Mar 19
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 19 March 2019 at 19:22:43 UTC, Walter Bright wrote:
 

 I've got dip1000 working with nearly all of Phobos now, which 
 is a crucial foundational step.
I hadn't realized that you were that far along. That's great.
Mar 19
parent walker <growup_wei qq.com> writes:
On Tuesday, 19 March 2019 at 19:30:53 UTC, jmh530 wrote:
 On Tuesday, 19 March 2019 at 19:22:43 UTC, Walter Bright wrote:
 

 I've got dip1000 working with nearly all of Phobos now, which 
 is a crucial foundational step.
I hadn't realized that you were that far along. That's great.
Maybe it is about the dip1000.mak https://github.com/dlang/phobos/blob/master/dip1000.mak
Mar 19
prev sibling parent reply Meta <jared771 gmail.com> writes:
On Tuesday, 19 March 2019 at 04:25:30 UTC, Walter Bright wrote:
 https://github.com/dlang/phobos/pull/6919

 The function returns a pointer to a local variable. The 
 pre-DIP1000 code circumvented the compiler error message by 
 passing the address through another variable. DIP1000 tracks 
 the lifetimes across assignments, exposing such shenanigans.

 These sorts of bugs can be very hard to track down by their 
 effects.
I think we have a ways to go with DIP1000/DIP25, but after actually putting them to use I am cautiously optimistic. I was expecting it to be a nightmare to annotate my recent pet project, but it actually wasn't too bad and only took an evening going from 0 to about 85% annotated. My main concern is that it doesn't seem to do a single thing if your functions aren't marked as safe; from what I can tell, `scope`, `return scope`, etc. are entirely ignored without that safe, which can be confusing when you're trying to figure out why something isn't working, only to realize you forgot safe. This is also a big plus, of course, and I attribute the relative ease of transitioning my codebase to this fact. It definitely makes the burden of compliance for large code bases much easier. Documentation is also a big concern, of course. That and some bugs (or perhaps just unsupported use cases - see my other thread) are two problems that I expect is just a matter of time and a larger number of people using DIP1000/DIP25, finding all the corner cases that need to be ironed out.
Mar 19
parent reply Walter Bright <newshound2 digitalmars.com> writes:
As of today I finally got all of Phobos to compile with DIP1000. The remaining 
issue is that things sometimes get mangled differently with DIP1000 than 
without, leading to linking errors.

But it's working with all compiled with -dip1000.
Mar 20
next sibling parent Andre Pany <andre s-e-a-p.de> writes:
On Wednesday, 20 March 2019 at 10:00:39 UTC, Walter Bright wrote:
 As of today I finally got all of Phobos to compile with 
 DIP1000. The remaining issue is that things sometimes get 
 mangled differently with DIP1000 than without, leading to 
 linking errors.

 But it's working with all compiled with -dip1000.
Sounds great. What is the plan regarding the managing issue? Kind regards Andre
Mar 20
prev sibling next sibling parent Jonathan Marler <johnnymarler gmail.com> writes:
On Wednesday, 20 March 2019 at 10:00:39 UTC, Walter Bright wrote:
 As of today I finally got all of Phobos to compile with 
 DIP1000. The remaining issue is that things sometimes get 
 mangled differently with DIP1000 than without, leading to 
 linking errors.

 But it's working with all compiled with -dip1000.
I was working through this issue a year ago. Created this thread to discuss potential solutions: "D Library Breakage" https://forum.dlang.org/thread/nbdzcwlgzvdcxnecefdk forum.dlang.org I'd like to see a way for D to be able to pull compilation settings from pre-compiled librarys. Settings like what versions they were compiled with, what optional features they used (i.e. dip1000). This way, the interpretation of the library code that gets imported always matches what's actually in the pre-compiled library. This could be done by adding that information to the library file or creating a new file that can be deployed alongside the library with those pre-compiled settings.
Mar 20
prev sibling parent reply bitwise <bitwise.pvt gmail.com> writes:
On Wednesday, 20 March 2019 at 10:00:39 UTC, Walter Bright wrote:
 As of today I finally got all of Phobos to compile with 
 DIP1000. The remaining issue is that things sometimes get 
 mangled differently with DIP1000 than without, leading to 
 linking errors.

 But it's working with all compiled with -dip1000.
Just out of curiosity - did that involve removing all instances of the 'in' type qualifier? I guess I'm still holding out hope that it will be rebranded as the counterpart of 'out', as a by-ref, in-only, parameter type that accepts rvalues ;) It's disappointing to hear the current plans for scope are so limited. I originally thought that scope would be a general type qualifier which restricted anything it was applied to from being copied out of it's current scope. I was hoping to be able to apply scope to range objects returned from containers so they couldn't leave the scope of where they were retrieved. I also saw that Atila used scope on a postblit, which also seems like something that should keep the range from leaving the scope where it was retrieved. If I understand correctly though, neither of these uses is officially supported or planned right now - is that correct? Thanks, Bit
Mar 23
parent reply Meta <jared771 gmail.com> writes:
On Saturday, 23 March 2019 at 16:32:00 UTC, bitwise wrote:
 It's disappointing to hear the current plans for scope are so 
 limited. I originally thought that scope would be a general 
 type qualifier which restricted anything it was applied to from 
 being copied out of it's current scope.

 I was hoping to be able to apply scope to range objects 
 returned from containers so they couldn't leave the scope of 
 where they were retrieved. I also saw that Atila used scope on 
 a postblit, which also seems like something that should keep 
 the range from leaving the scope where it was retrieved. If I 
 understand correctly though, neither of these uses is 
 officially supported or planned right now - is that correct?

 Thanks,
     Bit
I also had this misconception initially because that's how scope was originally defined to work; hence this thread: https://forum.dlang.org/thread/zayyoiupftrvbummxabk forum.dlang.org
Mar 23
next sibling parent bitwise <bitwise.pvt gmail.com> writes:
On Saturday, 23 March 2019 at 19:58:34 UTC, Meta wrote:
 On Saturday, 23 March 2019 at 16:32:00 UTC, bitwise wrote:
     [...]
I also had this misconception initially because that's how scope was originally defined to work; hence this thread: https://forum.dlang.org/thread/zayyoiupftrvbummxabk forum.dlang.org
It makes no sense to have one keyword that overlaps two others, especially when there's such a strong bias against adding new keywords.
Mar 25
prev sibling parent bitwise <bitwise.pvt gmail.com> writes:
On Saturday, 23 March 2019 at 19:58:34 UTC, Meta wrote:
 [...]
That was a serious question about "in". (this point was supposed to precede the last comment, but was eaten by the forum)
Mar 25