www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Call to Action: making Phobos safe

reply Walter Bright <newshound2 digitalmars.com> writes:
Andrei identified a key blocker for D adoption is the incomplete implementation 
of  safe. I'm working on the compiler end. But Phobos has a lot of code that is 
pointlessly not  safe, making it frustrating to write  safe code that calls 
Phobos. Some are listed in Bugzilla, most are not.

So here's what to do:

1. Pick a module, any module, let's say 'std.foo'.

2. Compile it:

     dmd -unittest -main std/foo

and run:

     foo

to verify that the unittests work.

3. Take a look at foo.d and look for a unittest that is not marked  safe.

4. Add  safe to the unittest

5. Do Step 2 on it.

6. If it works, submit a PR with the annotation (well, try to collect a few of 
these in one module)

7. If it doesn't work, find out where the unsafe code is and fix it, and submit 
a PR.


I've submitted maybe a dozen PRs against Phobos over the last day from doing 
this. It's only a start.
Jun 25 2016
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
Debugging tip:

Templates should almost never be marked  safe or  trusted, as the safety should 
be inferred. However, to isolate down where the unsafe code actually is, add 
 safe temporarily to the template declaration. Then, the compiler will tell you 
which line in the template is unsafe.
Jun 25 2016
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/25/2016 3:44 PM, Walter Bright wrote:
 4. Add  safe to the unittest
A unittest that is deliberately unsafe should be annotated with system. Meaning that any un-annotated unittest needs corrective action one way or the other.
Jun 25 2016
next sibling parent reply Tourist <gravatar gravatar.com> writes:
On Saturday, 25 June 2016 at 22:56:12 UTC, Walter Bright wrote:
 On 6/25/2016 3:44 PM, Walter Bright wrote:
 4. Add  safe to the unittest
A unittest that is deliberately unsafe should be annotated with system. Meaning that any un-annotated unittest needs corrective action one way or the other.
So why not declare safe module-wise, and only add system annotations where needed?
Jun 25 2016
parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sunday, June 26, 2016 06:29:05 Tourist via Digitalmars-d wrote:
 On Saturday, 25 June 2016 at 22:56:12 UTC, Walter Bright wrote:
 On 6/25/2016 3:44 PM, Walter Bright wrote:
 4. Add  safe to the unittest
A unittest that is deliberately unsafe should be annotated with system. Meaning that any un-annotated unittest needs corrective action one way or the other.
So why not declare safe module-wise, and only add system annotations where needed?
In general, IMHO, mass applying attributes is a bad idea, because it makes it harder to know which attributes apply where, which is a maintenance problem. However, even worse than that, while there it _is_ possible to apply safe to the module as a whole and then selectively apply trusted and system explicitly, there is no way to turn off all of the safety attributes so that attribute inference can be used. And other attributes can't be undone at all. So, if you have templates in a module, mass applying attributes is a disaster. - Jonathan M Davis
Jun 25 2016
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Saturday, 25 June 2016 at 22:56:12 UTC, Walter Bright wrote:
 On 6/25/2016 3:44 PM, Walter Bright wrote:
 4. Add  safe to the unittest
A unittest that is deliberately unsafe should be annotated with system. Meaning that any un-annotated unittest needs corrective action one way or the other.
What about existing tests that are marked trusted? Atila
Jun 29 2016
next sibling parent reply Lodovico Giaretta <lodovico giaretart.net> writes:
On Wednesday, 29 June 2016 at 17:25:31 UTC, Atila Neves wrote:
 On Saturday, 25 June 2016 at 22:56:12 UTC, Walter Bright wrote:
 On 6/25/2016 3:44 PM, Walter Bright wrote:
 4. Add  safe to the unittest
A unittest that is deliberately unsafe should be annotated with system. Meaning that any un-annotated unittest needs corrective action one way or the other.
What about existing tests that are marked trusted? Atila
Maybe I'm missing something, but... What's the purpose of a trusted unittest? If it's safe, mark it safe. If it's not, you gain nothing from marking it trusted, given that it will not be called by safe code (it will not be called in general). trusting a top-level function is useless IMHO...
Jun 29 2016
parent Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 29 June 2016 at 17:33:18 UTC, Lodovico Giaretta 
wrote:
 On Wednesday, 29 June 2016 at 17:25:31 UTC, Atila Neves wrote:
 On Saturday, 25 June 2016 at 22:56:12 UTC, Walter Bright wrote:
 On 6/25/2016 3:44 PM, Walter Bright wrote:
 4. Add  safe to the unittest
A unittest that is deliberately unsafe should be annotated with system. Meaning that any un-annotated unittest needs corrective action one way or the other.
What about existing tests that are marked trusted? Atila
Maybe I'm missing something, but... What's the purpose of a trusted unittest? If it's safe, mark it safe. If it's not, you gain nothing from marking it trusted, given that it will not be called by safe code (it will not be called in general). trusting a top-level function is useless IMHO...
I don't think there's any point either, I just thought I'd ask before changing them to system. Atila
Jun 29 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/29/2016 10:25 AM, Atila Neves wrote:
 On Saturday, 25 June 2016 at 22:56:12 UTC, Walter Bright wrote:
 On 6/25/2016 3:44 PM, Walter Bright wrote:
 4. Add  safe to the unittest
A unittest that is deliberately unsafe should be annotated with system. Meaning that any un-annotated unittest needs corrective action one way or the other.
What about existing tests that are marked trusted?
Marking a unittest as trusted seems pointless.
Jun 29 2016
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/29/16 4:57 PM, Walter Bright wrote:
 On 6/29/2016 10:25 AM, Atila Neves wrote:
 On Saturday, 25 June 2016 at 22:56:12 UTC, Walter Bright wrote:
 On 6/25/2016 3:44 PM, Walter Bright wrote:
 4. Add  safe to the unittest
A unittest that is deliberately unsafe should be annotated with system. Meaning that any un-annotated unittest needs corrective action one way or the other.
What about existing tests that are marked trusted?
Marking a unittest as trusted seems pointless.
I would further say -- if you see a trusted unit test, whoever wrote it doesn't understand the point of trusted. Be sure and check the code it is testing too, and file bugs if appropriate. -Steve
Jun 30 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/30/2016 4:09 AM, Steven Schveighoffer wrote:
 I would further say -- if you see a  trusted unit test, whoever wrote it
doesn't
 understand the point of  trusted. Be sure and check the code it is testing too,
 and file bugs if appropriate.
I agree. Using trusted properly turns out to have some surprising subtleties, but this one is pretty much a no-brainer.
Jun 30 2016
prev sibling next sibling parent Andre Pany <andre s-e-a-p.de> writes:
On Saturday, 25 June 2016 at 22:44:37 UTC, Walter Bright wrote:
 Andrei identified a key blocker for D adoption is the 
 incomplete implementation of  safe. I'm working on the compiler 
 end. But Phobos has a lot of code that is pointlessly not 
  safe, making it frustrating to write  safe code that calls 
 Phobos. Some are listed in Bugzilla, most are not.

 So here's what to do:

 1....
Could we have a wiki page for this call to action? It would be much easier if there is a list of the modules and their current state like: completed or who is working on this module. In general the call to actions topics disappears too fast in a forum. Maybe it worths to add them to the contributions group in the wiki? Kind regards Andre
Jun 26 2016
prev sibling next sibling parent reply Robert burner Schadek <rburners gmail.com> writes:
It would be awesome if you would create that process model in the 
wiki and at it to your action list 
http://wiki.dlang.org/Walter_Andrei_Action_List#Walter_and_Andrei.27s_Action_List

So it does not get lost and people can find it.
Jun 26 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/26/2016 6:13 AM, Robert burner Schadek wrote:
 It would be awesome if you would create that process model in the wiki and at
it
 to your action list
 http://wiki.dlang.org/Walter_Andrei_Action_List#Walter_and_Andrei.27s_Action_List
It's a wiki, feel free to add it.
Jun 26 2016
parent reply Robert burner Schadek <rburners gmail.com> writes:
On Sunday, 26 June 2016 at 22:38:54 UTC, Walter Bright wrote:
 It's a wiki, feel free to add it.
I have to say that reply really makes me angry. I created that list so Andrei and you have an easy to find spot where you can write down tasks so people can work on them. You did not disagree with the list at the time, and didn't disagree so far. I really think it is clear that these are two list that you two are maintaining, how else should anybody know what work is pre-approved and where D is going. I think I'm not asking too much here, if I ask you to decide if you want to maintain that list or if you want to delete it. The current limbo state is just making things worse.
Jun 27 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/27/2016 8:14 AM, Robert burner Schadek wrote:
 On Sunday, 26 June 2016 at 22:38:54 UTC, Walter Bright wrote:
 It's a wiki, feel free to add it.
I have to say that reply really makes me angry. I created that list so Andrei and you have an easy to find spot where you can write down tasks so people can work on them. You did not disagree with the list at the time, and didn't disagree so far. I really think it is clear that these are two list that you two are maintaining, how else should anybody know what work is pre-approved and where D is going. I think I'm not asking too much here, if I ask you to decide if you want to maintain that list or if you want to delete it. The current limbo state is just making things worse.
Sorry to have offended you, I worded things badly. Thank you for making the list. It's just that I'm feeling a bit overwhelmed at the moment with trying to get things done and being asked to do more every day, and I'd like to delegate.
Jun 27 2016
parent reply Robert burner Schadek <rburners gmail.com> writes:
On Monday, 27 June 2016 at 19:33:45 UTC, Walter Bright wrote:
 Sorry to have offended you, I worded things badly. Thank you 
 for making the list. It's just that I'm feeling a bit 
 overwhelmed at the moment with trying to get things done and 
 being asked to do more every day, and I'd like to delegate.
Don't sweat it. I just replied in the heat of the moment and thereby violated my personal 24 hour no reply rule. Back to topic, I think this list will save you and Andrei work in the long run. I mean if the list is up to date, you do not have to repeat yourself. Well expect "read the list!" ;-)
Jun 28 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 06/28/2016 05:34 PM, Robert burner Schadek wrote:
 On Monday, 27 June 2016 at 19:33:45 UTC, Walter Bright wrote:
 Sorry to have offended you, I worded things badly. Thank you for
 making the list. It's just that I'm feeling a bit overwhelmed at the
 moment with trying to get things done and being asked to do more every
 day, and I'd like to delegate.
Don't sweat it. I just replied in the heat of the moment and thereby violated my personal 24 hour no reply rule. Back to topic, I think this list will save you and Andrei work in the long run. I mean if the list is up to date, you do not have to repeat yourself. Well expect "read the list!" ;-)
That's a great rule. I should add that the official "things to do" list is the vision document. We don't need a separate list. -- Andrei
Jun 28 2016
prev sibling parent reply Bennet Leff <bennetleff gmail.com> writes:
On Sunday, 26 June 2016 at 13:13:01 UTC, Robert burner Schadek 
wrote:
 It would be awesome if you would create that process model in 
 the wiki and at it to your action list 
 http://wiki.dlang.org/Walter_Andrei_Action_List#Walter_and_Andrei.27s_Action_List

 So it does not get lost and people can find it.
Could you elaborate on list option 9 "create a module that enables code to be run on GPU." Wouldn't the Derelict OpenCL bindings satisfy that need?
Jun 30 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/30/2016 11:54 AM, Bennet Leff wrote:
 On Sunday, 26 June 2016 at 13:13:01 UTC, Robert burner Schadek wrote:
 It would be awesome if you would create that process model in the wiki and at
 it to your action list
 http://wiki.dlang.org/Walter_Andrei_Action_List#Walter_and_Andrei.27s_Action_List

 So it does not get lost and people can find it.
Could you elaborate on list option 9 "create a module that enables code to be run on GPU." Wouldn't the Derelict OpenCL bindings satisfy that need?
I don't know. John Colvin is working on this, he could give a far better appraisal of the state.
Jun 30 2016
parent reply Seb <seb wilzba.ch> writes:
On Thursday, 30 June 2016 at 21:31:25 UTC, Walter Bright wrote:
 On 6/30/2016 11:54 AM, Bennet Leff wrote:
 On Sunday, 26 June 2016 at 13:13:01 UTC, Robert burner Schadek 
 wrote:
 It would be awesome if you would create that process model in 
 the wiki and at
 it to your action list
 http://wiki.dlang.org/Walter_Andrei_Action_List#Walter_and_Andrei.27s_Action_List

 So it does not get lost and people can find it.
Could you elaborate on list option 9 "create a module that enables code to be run on GPU." Wouldn't the Derelict OpenCL bindings satisfy that need?
I don't know. John Colvin is working on this, he could give a far better appraisal of the state.
Some pointers: https://github.com/John-Colvin/clWrap http://dconf.org/2016/talks/colvin.html
Jun 30 2016
parent Bennet Leff <bennetleff gmail.com> writes:
On Thursday, 30 June 2016 at 21:55:33 UTC, Seb wrote:
 On Thursday, 30 June 2016 at 21:31:25 UTC, Walter Bright wrote:
 On 6/30/2016 11:54 AM, Bennet Leff wrote:
 On Sunday, 26 June 2016 at 13:13:01 UTC, Robert burner 
 Schadek wrote:
 [...]
Could you elaborate on list option 9 "create a module that enables code to be run on GPU." Wouldn't the Derelict OpenCL bindings satisfy that need?
I don't know. John Colvin is working on this, he could give a far better appraisal of the state.
Some pointers: https://github.com/John-Colvin/clWrap http://dconf.org/2016/talks/colvin.html
Thanks I'll take a further look although it doesn't look like his project is ready for more contributors to add to it yet.
Jul 01 2016
prev sibling next sibling parent "Smoke" Adams <SA2432 gmail.com> writes:
On Saturday, 25 June 2016 at 22:44:37 UTC, Walter Bright wrote:
 Andrei identified a key blocker for D adoption is the 
 incomplete implementation of  safe. I'm working on the compiler 
 end. But Phobos has a lot of code that is pointlessly not 
  safe, making it frustrating to write  safe code that calls 
 Phobos. Some are listed in Bugzilla, most are not.

 [...]
How hard would it be to add a compiler switch, similar to -vgs that shows every non-safe function? This would not just help out with phobos.
Jun 26 2016
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Saturday, 25 June 2016 at 22:44:37 UTC, Walter Bright wrote:
 Andrei identified a key blocker for D adoption is the 
 incomplete implementation of  safe. I'm working on the compiler 
 end. But Phobos has a lot of code that is pointlessly not 
  safe, making it frustrating to write  safe code that calls 
 Phobos. Some are listed in Bugzilla, most are not.

 So here's what to do:

 1. Pick a module, any module, let's say 'std.foo'.

 2. Compile it:

     dmd -unittest -main std/foo

 and run:

     foo

 to verify that the unittests work.

 3. Take a look at foo.d and look for a unittest that is not 
 marked  safe.

 4. Add  safe to the unittest

 5. Do Step 2 on it.

 6. If it works, submit a PR with the annotation (well, try to 
 collect a few of these in one module)

 7. If it doesn't work, find out where the unsafe code is and 
 fix it, and submit a PR.


 I've submitted maybe a dozen PRs against Phobos over the last 
 day from doing this. It's only a start.
It's possible that someone introduced new unittests that aren't explicitly safe or system, but with the exception of std.stream (because of imminent deprecation) all phobos unittests are now explicity tagged. So now, if you want to help make Phobos safe, the steps are: 1. Pick a module, any module 2. Search for a system unittest 3. Change system to safe 4. Figure out why it doesn't compile that way and either edit code or file bugs Atila
Jul 14 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/14/2016 8:57 AM, Atila Neves wrote:
 It's possible that someone introduced new unittests that aren't explicitly
 safe
 or  system, but with the exception of std.stream (because of imminent
 deprecation) all phobos unittests are now explicity tagged. So now, if you want
 to help make Phobos  safe, the steps are:

 1. Pick a module, any module
 2. Search for a  system unittest
 3. Change  system to  safe
 4. Figure out why it doesn't compile that way and either edit code or file bugs
Awesome, thank you!
Jul 15 2016