www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - scope for array parameters

reply Benjamin Thaut <code benjamin-thaut.de> writes:
To make functions definitions more expressive and give the compiler more 
information to optimize I propose to make array function parameters 
extendable with 'scope' such as follows:

size_t find(scope float[] haystack, float needle){ ... }

This would give the compiler the information that the array haystack 
only has to be valid as long as the scope of the function find is valid. 
As soon as the function scope is left, the array does not need to be 
valid any more. This would greatly help when programming without a GC to 
know, if the array one passes to the function will be saved internally 
or if it only needs to be valid for the duration in which the function 
is executed. Also it would allow the compiler to optimize array literals 
such as:

find([1,2,3,4,5], 5)

The function call would not allocate on the heap, but would allocate the 
array literal on the stack, as the compiler knows that is only has to be 
valid for the scope of find.

Passing a scope array to another function which does not have the scope 
annotation should be an error. Assining a a scope array to a non local 
variable should be an error too. You could go really fancy on checking 
scoped function parameters but for a start simple rules should be enough.

The same could be done with references and pointers. The purpose of this 
would be to allow the compiler to do more static checking on the usage 
of the passed arguments and make the function definitions contain 
information if the passed references will be saved internally or not.

I do understand that non-gc programming is not considered often for the 
D-Programming Language, but it would be nice if this idea does not get 
ignored completely.

What do you think about this idea?

Kind Regards
Benjamin Thaut
Sep 04 2012
next sibling parent reply =?ISO-8859-15?Q?Alex_R=F8nne_Petersen?= <alex lycus.org> writes:
On 04-09-2012 22:13, Benjamin Thaut wrote:

 To make functions definitions more expressive and give the compiler more
 information to optimize I propose to make array function parameters
 extendable with 'scope' such as follows:

 size_t find(scope float[] haystack, float needle){ ... }

 This would give the compiler the information that the array haystack
 only has to be valid as long as the scope of the function find is valid.
 As soon as the function scope is left, the array does not need to be
 valid any more. This would greatly help when programming without a GC to
 know, if the array one passes to the function will be saved internally
 or if it only needs to be valid for the duration in which the function
 is executed. Also it would allow the compiler to optimize array literals
 such as:

 find([1,2,3,4,5], 5)

 The function call would not allocate on the heap, but would allocate the
 array literal on the stack, as the compiler knows that is only has to be
 valid for the scope of find.

 Passing a scope array to another function which does not have the scope
 annotation should be an error. Assining a a scope array to a non local
 variable should be an error too. You could go really fancy on checking
 scoped function parameters but for a start simple rules should be enough.

 The same could be done with references and pointers. The purpose of this
 would be to allow the compiler to do more static checking on the usage
 of the passed arguments and make the function definitions contain
 information if the passed references will be saved internally or not.

 I do understand that non-gc programming is not considered often for the
 D-Programming Language, but it would be nice if this idea does not get
 ignored completely.

 What do you think about this idea?

 Kind Regards
 Benjamin Thaut


This is already what scope does today. See http://dlang.org/function.html:

"references in the parameter cannot be escaped (e.g. assigned to a 
global variable)"

It's just that the compiler doesn't actually enforce it fully.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org
Sep 04 2012
parent reply Benjamin Thaut <code benjamin-thaut.de> writes:
Am 04.09.2012 22:18, schrieb Alex Rønne Petersen:

 On 04-09-2012 22:13, Benjamin Thaut wrote:
 This is already what scope does today. See http://dlang.org/function.html:

 "references in the parameter cannot be escaped (e.g. assigned to a
 global variable)"

 It's just that the compiler doesn't actually enforce it fully.


Does the compiler already use this information to optimize array literals?

Kind Regards
Benjamin Thaut
Sep 04 2012
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, September 04, 2012 22:19:55 Benjamin Thaut wrote:

 Am 04.09.2012 22:18, schrieb Alex Rønne Petersen:

 On 04-09-2012 22:13, Benjamin Thaut wrote:
 This is already what scope does today. See http://dlang.org/function.html:
 
 "references in the parameter cannot be escaped (e.g. assigned to a
 global variable)"
 
 It's just that the compiler doesn't actually enforce it fully.


 
 Does the compiler already use this information to optimize array literals?


It had better not (I doubt that it does though). Unless scope is actually 
properly enforced, then you're going to have major bugs if you use scope and 
then escape references to that data anyway. And without the compiler checks, 
that _will_ happen (especially when some folks use in all over the place).

Once scope is properly enforced, then optimizing based on it would be great, 
but until it is, it's a _bad_ idea.

- Jonathan M Davis
Sep 04 2012
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Jonathan M Davis:


 Once scope is properly enforced, then optimizing based on it 
 would be great, but until it is, it's a _bad_ idea.


Today we use "in" often for function arguments, and it implies 
"scope". Surely some of those programs use the arguments wrongly, 
this means they probably sometime escape. So are those programs 
someday going to become compilation errors? For the "-property" 
and for the arguments of delegate/function pointers the idea of 
turning some code currently used in errors is failing...
Such things need to be implemented early in a language.

Bye,
bearophile
Sep 04 2012
next sibling parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Tuesday, 4 September 2012 at 21:41:58 UTC, bearophile wrote:

 Jonathan M Davis:


 Once scope is properly enforced, then optimizing based on it 
 would be great, but until it is, it's a _bad_ idea.


 Today we use "in" often for function arguments, and it implies 
 "scope". Surely some of those programs use the arguments 
 wrongly, this means they probably sometime escape. So are those 
 programs someday going to become compilation errors?


Yep.


 Such things need to be implemented early in a language.


Yep.

It's very unfortunate because it puts us in a lose-lose 
situations where we can't fix the language without breaking 
people's code. I think the best course of action is to fix these 
things as soon as possible and get the problem out of the way.
Sep 04 2012
parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, September 04, 2012 23:57:24 Peter Alexander wrote:

 It's very unfortunate because it puts us in a lose-lose
 situations where we can't fix the language without breaking
 people's code. I think the best course of action is to fix these
 things as soon as possible and get the problem out of the way.


Indeed. The same goes for stuff like deprecating things that we've long said 
was going to be deprecated but hasn't yet been deprecated (e.g. delete). Lots 
of code is going to break when it happens, and the longer that we wait, the 
worse it's going to be. But until someone takes the time to fix it (and Walter 
is constantly busy on other things), it's not going to happen, which is 
obviously a big problem.

- Jonathan M Davis
Sep 04 2012
prev sibling next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, September 04, 2012 23:42:33 bearophile wrote:

 Jonathan M Davis:

 Once scope is properly enforced, then optimizing based on it
 would be great, but until it is, it's a _bad_ idea.


 
 Today we use "in" often for function arguments, and it implies
 "scope". Surely some of those programs use the arguments wrongly,
 this means they probably sometime escape. So are those programs
 someday going to become compilation errors? For the "-property"
 and for the arguments of delegate/function pointers the idea of
 turning some code currently used in errors is failing...
 Such things need to be implemented early in a language.


That's part of why I keep saying not to use in whenever it comes up. scope is 
very broken, so in is very broken. And honestly, given how often arrays are 
used in structs, I suspect that it's not at all uncommon for in to be used 
incorrectly.

I'd actually argue to _never_ to use in at this point. If you want const, then 
use const. If you want scope, then use scope. But using scope on anything 
which isn't checked properly by the compiler is just asking for it. You _will_ 
have code that breaks when scope is fixed. I believe that the only case that 
has _any_ protection at all with scope right now is delegates, which almost 
never should be const. So, using in makes little to no sense to me.

- Jonathan M Davis
Sep 04 2012
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Jonathan M Davis:


 That's part of why I keep saying not to use in whenever it 
 comes up. scope is
 very broken, so in is very broken. And honestly, given how 
 often arrays are
 used in structs, I suspect that it's not at all uncommon for in 
 to be used incorrectly.


The situation with "in"/"scope" is worse than just deprecated 
stuff like "delete" or "typedef". I know those things are going 
away, so I don't use them, and this avoids the problem.



 I believe that the only case that
 has _any_ protection at all with scope right now is delegates, 
 which almost never should be const.


Do you mean code like this? What's bad about this? My delegate 
arguments
/function pointer arguments are usually const.

void foo(const int delegate(int) dg) {}
void main() {
     foo((int x) => x);
}

Bye,
bearophile
Sep 04 2012
next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, September 05, 2012 01:50:12 bearophile wrote:

 I believe that the only case that
 has _any_ protection at all with scope right now is delegates,
 which almost never should be const.


 
 Do you mean code like this? What's bad about this? My delegate
 arguments
 /function pointer arguments are usually const.
 
 void foo(const int delegate(int) dg) {}
 void main() {
 foo((int x) => x);
 }


If you have delegates that work with const, then great, but in my experience 
(which frequently involves storing the delegate as a member variable), const 
doesn't play nicely at all with delegates.

- Jonathan M Davis
Sep 04 2012
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 09/05/2012 01:50 AM, bearophile wrote:

 Jonathan M Davis:


 That's part of why I keep saying not to use in whenever it comes up.
 scope is
 very broken, so in is very broken. And honestly, given how often
 arrays are
 used in structs, I suspect that it's not at all uncommon for in to be
 used incorrectly.


 The situation with "in"/"scope" is worse than just deprecated stuff like
 "delete" or "typedef". I know those things are going away, so I don't
 use them, and this avoids the problem.



 I believe that the only case that
 has _any_ protection at all with scope right now is delegates, which
 almost never should be const.


 Do you mean code like this? What's bad about this?
 My delegate arguments
 /function pointer arguments are usually const.

 void foo(const int delegate(int) dg) {}
 void main() {
      foo((int x) => x);
 }

 Bye,
 bearophile


Similar code is the main reason for the hole in the const system.
Some people want that fixed, ergo it might break.
Sep 05 2012
prev sibling parent reply "ixid" <nuaccount gmail.com> writes:
What does -property supposedly solve? It creates a horrid mess of 
brackets that ruin the elegance of UFCS code.
Sep 04 2012
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, September 05, 2012 01:13:00 ixid wrote:

 What does -property supposedly solve?


It's supposed to make it so that property functions are used as variables and 
non-property functions are used as functions, since the point of property 
functions is to emulate variables. It's very buggy at the moment though, so it 
doesn't really enforce everything that it's supposed to enforce. IIRC, it only 
enforces that non-property functions are called with parens and not that 
property functions are called without them.


 It creates a horrid mess of
 brackets that ruin the elegance of UFCS code.


You mean parens? Regardless, it means that you using functions as if there 
were variables, which I'm very much against. They're functions and should be 
called as such. But not everyone agrees.

I think that it's pretty much a guarantee that we're eventually going to end 
up with  property functions having to be called without parens (I don't think 
that very many people disagree on that). The bigger question is whether we're 
going to follow TDPL and also make it so that non-property functions must be 
called with them (as -property currently checks for). The number of parens 
required with UFCS (particularly with functions requiring template arguments) 
is one of the reasons that some people give that they think that non-property 
functions shouldn't be forced to be called with parens.

- Jonathan M Davis
Sep 04 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09/05/2012 02:10 AM, Jonathan M Davis wrote:

 On Wednesday, September 05, 2012 01:13:00 ixid wrote:

 What does -property supposedly solve?


 It's supposed to make it so that property functions are used as variables and
 non-property functions are used as functions, since the point of property
 functions is to emulate variables. It's very buggy at the moment though, so it
 doesn't really enforce everything that it's supposed to enforce. IIRC, it only
 enforces that non-property functions are called with parens and not that
 property functions are called without them.


 It creates a horrid mess of
 brackets that ruin the elegance of UFCS code.


 You mean parens? Regardless, it means that you using functions as if there
 were variables, which I'm very much against. They're functions and should be
 called as such. But not everyone agrees.

 I think that it's pretty much a guarantee that we're eventually going to end
 up with  property functions having to be called without parens (I don't think
 that very many people disagree on that). The bigger question is whether we're
 going to follow TDPL and also make it so that non-property functions must be
 called with them (as -property currently checks for).


TDPL does not prescribe this.


 The number of parens
 required with UFCS (particularly with functions requiring template arguments)
 is one of the reasons that some people give that they think that non-property
 functions shouldn't be forced to be called with parens.

 - Jonathan M Davis
Sep 05 2012
parent "Chris Nicholson-Sauls" <ibisbasenji gmail.com> writes:
On Wednesday, 5 September 2012 at 12:05:40 UTC, Timon Gehr wrote:

 TDPL does not prescribe this.


Truth; I just checked this and indeed TDPL only mentions 
 property functions disavowing themselves of parentheses.  
Although I've always had the distinct impression that strict 
property syntax was the intended future, and the current 
situation was just due to inheriting D1's clumsier 'property' 
concept.  I don't even mind all those "ugly" parens in UFCS 
chains, but I'm kinda crazy like that.
Sep 05 2012
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, September 04, 2012 22:13:44 Benjamin Thaut wrote:

 What do you think about this idea?


scope on function parameters is supposed to prevent any reference to that data 
escaping from the function. e.g. scope on a delegate would make it an error to 
assign that delegate to anything which could escape the function. It is my 
understanding that this is also supposed to include arrays such that if you 
mark an array parameter with scope, it should be illegal for any references to 
it (including slices of it or references to its elements) to escape the 
function, which _would_ allow compiler optimizations. But the compiler doesn't 
seem to properly check for much with regards to scope right now. A _lot_ of 
code is going to break once it does, particularly since some people seem to 
think that it's a good idea to use in (which is the same as const scope) 
everywhere.

- Jonathan M Davis
Sep 04 2012