www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Puzzled by this behavior

reply Don Allen <donaldcallen gmail.com> writes:
This

````
import std.stdio;

void foo() {
     bar();
}
void bar() {
     writeln("Hello world");
}
int main(string[] args)
{
     foo();
     return 0;
}
````

compiles and does what you'd expect, despite the forward 
reference from foo to bar, which, of course, would not work in C.

But
````
import std.stdio;

int main(string[] args)
{
     void foo() {
         bar();
     }
     void bar() {
         writeln("Hello world");
     }
     foo();
     return 0;
}
````
gives the error
````
(dmd-2.100.0)dca pangloss.allen.net:/home/dca/Software/d_tests$ 
dmd test5.d
test5.d(6): Error: undefined identifier `bar`
````
This only works if you interchange the order of foo and bar, 
eliminating the
forward reference.

This strikes me as pretty inconsistent behavior, especially to 
someone who has written as much Scheme and Haskell as I have. 
I've also not found it documented, though I could have missed it 
(if someone could point me to where this is discussed, I'd 
appreciate it). My main purpose in sending this message is to 
understand whether this is a bug or a documented "feature". If 
the former, I will file a bug report.

/Don
May 31 2022
next sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 5/31/22 10:41, Don Allen wrote:


 I've also not
 found it documented


This is by-design. One place I found is the following FAQ entry:

   https://dlang.org/articles/faq.html#nested_forward_references

Ali
May 31 2022
prev sibling next sibling parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Tuesday, 31 May 2022 at 17:41:18 UTC, Don Allen wrote:

 This strikes me as pretty inconsistent behavior


Code in functions is actually executed in sequence. Nested 
functions aren't exactly code, but the same rule applies to them. 
Consider:

int a = 5;
a = 6;
int b = a;

What is b? Of course we know since it happens in sequence. Same 
rule applies with nested functions.


 I've also not found it documented, though I could have missed 
 it (if someone could point me to where this is discussed, I'd


It has its own section on the function page:

https://dlang.org/spec/function.html#nested-declaration-order
May 31 2022
parent reply Don Allen <donaldcallen gmail.com> writes:
On Tuesday, 31 May 2022 at 17:52:35 UTC, Adam D Ruppe wrote:

 On Tuesday, 31 May 2022 at 17:41:18 UTC, Don Allen wrote:

 This strikes me as pretty inconsistent behavior


 Code in functions is actually executed in sequence. Nested 
 functions aren't exactly code, but the same rule applies to 
 them. Consider:

 int a = 5;
 a = 6;
 int b = a;

 What is b? Of course we know since it happens in sequence. Same 
 rule applies with nested functions.


 I've also not found it documented, though I could have missed 
 it (if someone could point me to where this is discussed, I'd


 It has its own section on the function page:

 https://dlang.org/spec/function.html#nested-declaration-order


Code in Scheme functions are also evaluated in sequence, but 
functions can be mutually recursive whether at top-level or not, 
so the mere fact of sequential evaluation is not the explanation. 
What appears to matter is when the location of called functions 
are resolved -- compile-time (and if so, what pass?) or run-time. 
  In Scheme and Haskell, it is (at least conceptually) at 
run-time. D appears to do it both ways, depending on the 
particular situation (top-level? nested? static methods?). 
Without understanding the reason for this choice, I must say I 
find it quite odd.
May 31 2022
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 31 May 2022 at 18:30:12 UTC, Don Allen wrote:

 static methods?). Without understanding the reason for this 
 choice, I must say I find it quite odd.


The reason is that D is more evolved than designed. There is no 
reason for mixing declaration order with checking for 
initialization before use.

This works with lambdas:

```
void main()
{
     int i;
     void delegate() foo;

     auto bar = (){
         writeln("Hello world");
         if(++i < 10) foo();
     };

     foo = (){ bar(); };

     foo();
}
```
May 31 2022
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 5/31/22 2:30 PM, Don Allen wrote:

 On Tuesday, 31 May 2022 at 17:52:35 UTC, Adam D Ruppe wrote:

 On Tuesday, 31 May 2022 at 17:41:18 UTC, Don Allen wrote:

 This strikes me as pretty inconsistent behavior


 Code in functions is actually executed in sequence. Nested functions 
 aren't exactly code, but the same rule applies to them. Consider:

 int a = 5;
 a = 6;
 int b = a;

 What is b? Of course we know since it happens in sequence. Same rule 
 applies with nested functions.


 I've also not found it documented, though I could have missed it (if 
 someone could point me to where this is discussed, I'd


 It has its own section on the function page:

 https://dlang.org/spec/function.html#nested-declaration-order


 
 Code in Scheme functions are also evaluated in sequence, but functions 
 can be mutually recursive whether at top-level or not, so the mere fact 
 of sequential evaluation is not the explanation.


The scheme code likely evaluates the *definition* of the function, 
without resolving what *code* to call until it encounters a call.

In other words, something like this in D (I haven't used scheme in a 
while, so we are going to use D syntax):

```d
void foo() { bar(); }
void bar() { writeln("hello"); }
foo();
```

would work with scheme rules but

```d
void foo() { bar(); }
foo();
void bar() { writeln("hello"); }
```

would not.


 What appears to matter 
 is when the location of called functions are resolved -- compile-time 


It's not a matter of when they are resolved. D is fully capable of 
resolving functions at compile time out of order. It's a question of 
what symbols *are in scope*.

Consider:

```d
void foo() { writeln("outer foo"); }


void main()
{
    void bar() { foo(); }
    void foo() { writeln("inner foo"); }
    bar();
}
```

what should print is "outer foo", because *that* is what `foo` means at 
the point in which bar is being compiled. Inside functions, order of 
declaration is important and significant. Outside functions, they can be 
in any order, but must not be ambiguous. These are incompatible sets of 
rules. You have to pick one, and D chose to pick C rules inside 
functions (likely for compatibility), but allowed out of order 
declarations outside them because prototyping is just monotonous.

This is done on purpose. The easiest way to solve this is to declare the 
functions inside a struct in the function (as the workarounds suggest). 
I've done this many times when I have a significantly complex recursive 
algorithm that I don't want to expose outside the function.

Note that you can declare a prototype, but this also declares a symbol, 
and D does not allow you to redefine symbols.

-Steve
May 31 2022
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 31 May 2022 at 19:59:21 UTC, Steven Schveighoffer 
wrote:

 Note that you can declare a prototype, but this also declares a 
 symbol, and D does not allow you to redefine symbols.


And this doesn't make any sense. Why would local functions not 
work like lambdas?
May 31 2022
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 5/31/22 4:11 PM, Ola Fosheim Grøstad wrote:

 On Tuesday, 31 May 2022 at 19:59:21 UTC, Steven Schveighoffer wrote:

 Note that you can declare a prototype, but this also declares a 
 symbol, and D does not allow you to redefine symbols.


 
 And this doesn't make any sense. Why would local functions not work like 
 lambdas?


Because they aren't lambdas. To use your example:

```d
void main()
{
     int i;
     void foo(); // declares foo with no definition

     auto bar = (){
         writeln("Hello world");
         if(++i < 10) foo();
     };

     void foo() {bar(); }; // redeclares foo, not allowed

     foo();
}
```

would be the same with lambdas as:

```d
void main()
{
     int i;
     void delegate() foo; // declares foo and assigns it to null

     auto bar = (){
         writeln("Hello world");
         if(++i < 10) foo();
     };

     //foo = (){ bar(); }; // this is NOT a declaration, it's an assignment
     void delegate() foo = () {bar(); }; // redeclares foo, not allowed

     foo();
}
```

You can't "assign" functions like you can lambdas. And D does not allow 
redeclaring a symbol of any type in a specific scope.

-Steve
May 31 2022
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 31 May 2022 at 20:24:17 UTC, Steven Schveighoffer 
wrote:


 ```d
     void foo() {bar(); }; // redeclares foo, not allowed
 ```


This ought to define what has already been declared. That is the 
purpose of a prototype. Calling this a "redeclaration" is 
arbitrary.



 would be the same with lambdas as:

 ```d
     void delegate() foo; // declares foo and assigns it to null
```


This declares and defines foo.


 You can't "assign" functions like you can lambdas. And D does 
 not allow redeclaring a symbol of any type in a specific scope.


Why do you call it "redeclaring" the signature is the same!? 
That's just an after-the-fact explanation. Calling this 
redeclaring doesn't follow from how prototypes in C works. If the 
signature is the same then you can do it as many times as you 
want.
May 31 2022
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 5/31/22 4:33 PM, Ola Fosheim Grøstad wrote:

 On Tuesday, 31 May 2022 at 20:24:17 UTC, Steven Schveighoffer wrote:
 

 ```d
     void foo() {bar(); }; // redeclares foo, not allowed
 ```


 
 This ought to define what has already been declared. That is the purpose 
 of a prototype. Calling this a "redeclaration" is arbitrary.


I don't know why it does that, I assume Walter has a good reason for 
that. But they do work at module level (which is kinda weird, since you 
don't need them there). I had thought they weren't allowed.


 You can't "assign" functions like you can lambdas. And D does not 
 allow redeclaring a symbol of any type in a specific scope.


 
 Why do you call it "redeclaring" the signature is the same!? That's just 
 an after-the-fact explanation. Calling this redeclaring doesn't follow 
 from how prototypes in C works. If the signature is the same then you 
 can do it as many times as you want.


I can't do:

```d
int x;
int x = 5;
```

So I assume it's similar to declaring functions, but I don't know the 
reason it allows this for function prototypes at the module level.

-Steve
May 31 2022
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 31 May 2022 at 20:47:36 UTC, Steven Schveighoffer 
wrote:

 So I assume it's similar to declaring functions, but I don't 
 know the reason it allows this for function prototypes at the 
 module level.


Maybe he didn't think about mutual recursion.

Anyway, this is legal in C++:

```
extern int x;
extern int x;
void f();
void f();
void f(){ x=0;}
int x = 1;
```
May 31 2022
prev sibling next sibling parent reply Max Samukha <maxsamukha gmail.com> writes:
On Tuesday, 31 May 2022 at 19:59:21 UTC, Steven Schveighoffer 
wrote:


 This is done on purpose.


What's the purpose? It looks like just another atavism, 
reproducing itself without any purpose.
May 31 2022
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 5/31/22 4:13 PM, Max Samukha wrote:

 On Tuesday, 31 May 2022 at 19:59:21 UTC, Steven Schveighoffer wrote:
 

 This is done on purpose.


 
 What's the purpose? It looks like just another atavism, reproducing 
 itself without any purpose.


To make code that is ported from C compile the same as it does in C.

-Steve
May 31 2022
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 31.05.22 22:24, Steven Schveighoffer wrote:

 On 5/31/22 4:13 PM, Max Samukha wrote:

 On Tuesday, 31 May 2022 at 19:59:21 UTC, Steven Schveighoffer wrote:


 This is done on purpose.


 What's the purpose? It looks like just another atavism, reproducing 
 itself without any purpose.


 
 To make code that is ported from C compile the same as it does in C.
 
 -Steve


There are local functions in C?
May 31 2022
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 5/31/22 8:17 PM, Timon Gehr wrote:

 On 31.05.22 22:24, Steven Schveighoffer wrote:

 On 5/31/22 4:13 PM, Max Samukha wrote:

 On Tuesday, 31 May 2022 at 19:59:21 UTC, Steven Schveighoffer wrote:


 This is done on purpose.


 What's the purpose? It looks like just another atavism, reproducing 
 itself without any purpose.


 To make code that is ported from C compile the same as it does in C.


 
 There are local functions in C?


I mean the lookup mechanisms are the same.

But yeah, there are no local functions in C. I just figured that this is 
so the compiler doesn't have to have weird special cases for lookups.

However, thinking about it more, we *do* allow function prototypes as 
local functions. But I can't figure out a way to actually define them, 
aside from using pragma(mangle).

I think there's an opportunity here where we can allow function 
prototypes, allow definitions later, and not break existing code.

-Steve
May 31 2022
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 5/31/22 8:29 PM, Steven Schveighoffer wrote:


 However, thinking about it more, we *do* allow function prototypes as 
 local functions. But I can't figure out a way to actually define them, 
 aside from using pragma(mangle).
 
 I think there's an opportunity here where we can allow function 
 prototypes, allow definitions later, and not break existing code.


Oh my, I just realized, a function prototype that's later defined could 
end up using stack frame data that doesn't exist. This is the reason for 
not allowing prototypes or forward reference functions:

```d
void foo()
{
   void bar();
   bar(); // would affect x before it exists
   int x = 5;
   void bar() { ++x; }
}
```

So just use a static struct if you want mutual recursion.

-Steve
May 31 2022
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 01.06.22 02:38, Steven Schveighoffer wrote:

 
 So just use a static struct if you want mutual recursion.


A static struct defeats much of the purpose of local functions.
May 31 2022
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 5/31/22 9:20 PM, Timon Gehr wrote:

 On 01.06.22 02:38, Steven Schveighoffer wrote:

 So just use a static struct if you want mutual recursion.


 
 A static struct defeats much of the purpose of local functions.


Sorry, I meant struct static functions. A static struct is OK if you 
just want recursive behavior, but don't need to access the stack frame. 
You can also use normal member functions inside a normal struct if you 
do need the stack frame.

I use it when I need to implement algorithms that are recursive, but I 
want to separate out the pieces to multiple functions.

-Steve
May 31 2022
prev sibling parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Wednesday, 1 June 2022 at 00:17:50 UTC, Timon Gehr wrote:

 There are local functions in C?


[Yes](https://gcc.gnu.org/onlinedocs/gcc-12.1.0/gcc/Nested-Functions.html#Nested-Functions).
Jun 03 2022
parent deadalnix <deadalnix gmail.com> writes:
On Friday, 3 June 2022 at 14:46:29 UTC, Iain Buclaw wrote:

 On Wednesday, 1 June 2022 at 00:17:50 UTC, Timon Gehr wrote:

 There are local functions in C?


 [Yes](https://gcc.gnu.org/onlinedocs/gcc-12.1.0/gcc/Nested-Functions.html#Nested-Functions).


Indeed, both GCC and clang implement this, but it's non standard.

The way it is implemented is really interesting, as it doesn't 
generate delegate, but proper function pointers with JITted 
trampolines.
Jun 03 2022
prev sibling parent reply Don Allen <donaldcallen gmail.com> writes:
On Tuesday, 31 May 2022 at 19:59:21 UTC, Steven Schveighoffer 
wrote:

 On 5/31/22 2:30 PM, Don Allen wrote:

 On Tuesday, 31 May 2022 at 17:52:35 UTC, Adam D Ruppe wrote:

 On Tuesday, 31 May 2022 at 17:41:18 UTC, Don Allen wrote:

 This strikes me as pretty inconsistent behavior


 Code in functions is actually executed in sequence. Nested 
 functions aren't exactly code, but the same rule applies to 
 them. Consider:

 int a = 5;
 a = 6;
 int b = a;

 What is b? Of course we know since it happens in sequence. 
 Same rule applies with nested functions.


 I've also not found it documented, though I could have 
 missed it (if someone could point me to where this is 
 discussed, I'd


 It has its own section on the function page:

 https://dlang.org/spec/function.html#nested-declaration-order


 
 Code in Scheme functions are also evaluated in sequence, but 
 functions can be mutually recursive whether at top-level or 
 not, so the mere fact of sequential evaluation is not the 
 explanation.


 The scheme code likely evaluates the *definition* of the 
 function, without resolving what *code* to call until it 
 encounters a call.

 In other words, something like this in D (I haven't used scheme 
 in a while, so we are going to use D syntax):

 ```d
 void foo() { bar(); }
 void bar() { writeln("hello"); }
 foo();
 ```

 would work with scheme rules but

 ```d
 void foo() { bar(); }
 foo();
 void bar() { writeln("hello"); }
 ```

 would not.


 What appears to matter is when the location of called 
 functions are resolved -- compile-time


 It's not a matter of when they are resolved. D is fully capable 
 of resolving functions at compile time out of order. It's a 
 question of what symbols *are in scope*.

 Consider:

 ```d
 void foo() { writeln("outer foo"); }


 void main()
 {
    void bar() { foo(); }
    void foo() { writeln("inner foo"); }
    bar();
 }
 ```

 what should print is "outer foo", because *that* is what `foo` 
 means at the point in which bar is being compiled.


Because D has chosen to define its version of lexical scoping in 
this way in this case, but not other cases. What you say would 
not be controversial if we were talking about
````
      int bar = foo + 1;
      int foo = 0;
````
That doesn't work in D, Scheme or Haskell, nor should it, because 
the value of a variable not yet defined is required in the course 
of sequential evaluation of the statements above.

But in the situation I encountered, the sequential evaluation is 
of function definitions. The forward reference is within one of 
those definitions and the actual reference does not occur until 
the function is invoked, at which point both functions have been 
defined and therefore I would expect foo and bar to be available 
to each other. What D is doing here makes no sense to me and is 
particularly bizarre because it does the opposite at top level 
and inside structs.


 Inside functions, order of declaration is important and 
 significant. Outside functions, they can be in any order, but 
 must not be ambiguous. These are incompatible sets of rules. 
 You have to pick one, and D chose to pick C rules inside 
 functions (likely for compatibility), but allowed out of order 
 declarations outside them because prototyping is just 
 monotonous.


Well, I would argue that that kind of thinking leads to a 
language that is a collection of special cases, rather than a 
language built on a core set of principles consistently applied.

Ok. I've had my say. I think this is a mistake, but I don't 
expect it to change because of my objection, so no point in 
pursuing.


 This is done on purpose. The easiest way to solve this is to 
 declare the functions inside a struct in the function (as the 
 workarounds suggest). I've done this many times when I have a 
 significantly complex recursive algorithm that I don't want to 
 expose outside the function.

 Note that you can declare a prototype, but this also declares a 
 symbol, and D does not allow you to redefine symbols.


My concern is not "solving this". My concern is whether the 
language is clean and consistent so I can have mental model of it 
I can rely upon, rather than constantly searching through 
documentation to learn how a particular special case is handled. 
I also care a lot about writing readable code, and about being 
able to limit the visibility of variables to just the scope that 
needs them and no more. The workarounds you cite frankly feel 
like hacks to me, ugly ways of working around problems in the 
language design.

/Don


 -Steve
May 31 2022
next sibling parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Tuesday, 31 May 2022 at 22:48:55 UTC, Don Allen wrote:

 My concern is not "solving this". My concern is whether the 
 language is clean and consistent so I can have mental model of 
 it I can rely upon, rather than constantly searching through 
 documentation to learn how a particular special case is 
 handled. I also care a lot about writing readable code, and 
 about being able to limit the visibility of variables to just 
 the scope that needs them and no more. The workarounds you cite 
 frankly feel like hacks to me, ugly ways of working around 
 problems in the language design.

 /Don


 -Steve




IMO this is completely internally consistent. Your mental model 
of a function should be a tree of scopes; your mental model of a 
module should be a lazily resolved key-value store. They're just 
fundamentally different things.

You can make function scopes behave similar to global scopes, ie. 
void delegate() foo; void bar() { foo(); } foo = { ... };, but 
then you just move the compiletime error to runtime: ie. void 
delegate() foo; foo(); You're basically manually implementing 
dynamic scoping. But D is lexically scoped for functions, not 
just with nested functions but with every language element. 
*Modules* are the exception - or rather, modules are just a 
fundamentally different thing.
Jun 01 2022
next sibling parent reply Ola Fosheim Gr <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 1 June 2022 at 07:54:04 UTC, FeepingCreature wrote:

 implementing dynamic scoping. But D is lexically scoped for 
 functions, not just with nested functions but with every 
 language element.


The functions foo and bar are in the same scope, checking for 
initialization before use is not strictly related. Also goto can 
jump to a forward label, so C allows some forward references.
Jun 01 2022
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 1 June 2022 at 08:22:24 UTC, Ola Fosheim Gr wrote:

 On Wednesday, 1 June 2022 at 07:54:04 UTC, FeepingCreature 
 wrote:

 implementing dynamic scoping. But D is lexically scoped for 
 functions, not just with nested functions but with every 
 language element.


 The functions foo and bar are in the same scope, checking for 
 initialization before use is not strictly related. Also goto 
 can jump to a forward label, so C allows some forward 
 references.


Well, I guess you can say that every new name creates a new scope 
as it shadows the outer scope in order. Anyway, a prototype isn't 
a bad trade-off.

(Btw IIRC «dynamic scope» is a completely different concept, that 
means the lookup is done down the call-graph. E.g. if function 
f() calls g() and f() has ```x```then g() will access f()'s 
```x``` when mentioning ```x```.)
Jun 01 2022
prev sibling next sibling parent Don Allen <donaldcallen gmail.com> writes:
On Wednesday, 1 June 2022 at 07:54:04 UTC, FeepingCreature wrote:

 On Tuesday, 31 May 2022 at 22:48:55 UTC, Don Allen wrote:

 My concern is not "solving this". My concern is whether the 
 language is clean and consistent so I can have mental model of 
 it I can rely upon, rather than constantly searching through 
 documentation to learn how a particular special case is 
 handled. I also care a lot about writing readable code, and 
 about being able to limit the visibility of variables to just 
 the scope that needs them and no more. The workarounds you 
 cite frankly feel like hacks to me, ugly ways of working 
 around problems in the language design.

 /Don


 -Steve




 IMO this is completely internally consistent. Your mental model 
 of a function should be a tree of scopes; your mental model of 
 a module should be a lazily resolved key-value store. They're 
 just fundamentally different things.

 You can make function scopes behave similar to global scopes, 
 ie. void delegate() foo; void bar() { foo(); } foo = { ... };, 
 but then you just move the compiletime error to runtime: ie. 
 void delegate() foo; foo(); You're basically manually 
 implementing dynamic scoping. But D is lexically scoped for 
 functions, not just with nested functions but with every 
 language element. *Modules* are the exception - or rather, 
 modules are just a fundamentally different thing.


I don't disagree with your description of D -- that module-level 
scopes are different than those at function level. You omitted 
struct-level scoping, which is similar to how modules are 
handled. And your discussion of how to make function-level 
scoping behave like module-level scoping ignores what I have 
previously written. I know of the availability of these 
workarounds and to me they are just band-aids on an odd language 
inconsistency.

I will say this once more and I'm done: function-level scoping in 
D is different from the other two D cases and different from 
other prominent lexically scoped languages. I question why that 
is. You assert that things are "internally" consistent. That may 
be so, but I would argue that that is completely irrelevant. What 
matters is *external* consistency, what the programmer sees.
Jun 01 2022
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 01.06.22 09:54, FeepingCreature wrote:

 On Tuesday, 31 May 2022 at 22:48:55 UTC, Don Allen wrote:

 My concern is not "solving this". My concern is whether the language 
 is clean and consistent so I can have mental model of it I can rely 
 upon, rather than constantly searching through documentation to learn 
 how a particular special case is handled. I also care a lot about 
 writing readable code, and about being able to limit the visibility of 
 variables to just the scope that needs them and no more. The 
 workarounds you cite frankly feel like hacks to me, ugly ways of 
 working around problems in the language design.

 /Don


 -Steve




 
 IMO this is completely internally consistent. Your mental model of a 
 function should be a tree of scopes; your mental model of a module 
 should be a lazily resolved key-value store. They're just fundamentally 
 different things.
 
 You can make function scopes behave similar to global scopes, ie. void 
 delegate() foo; void bar() { foo(); } foo = { ... };, but then you just 
 move the compiletime error to runtime: ie. void delegate() foo; foo(); 
 You're basically manually implementing dynamic scoping. But D is 
 lexically scoped for functions, not just with nested functions but with 
 every language element. *Modules* are the exception - or rather, modules 
 are just a fundamentally different thing.


The simple fact is that the current behavior is less useful than what we 
could very easily have. There is no need at all to solve this with 
function pointers the way you describe, just allow forward references 
and overloading between local functions that are declared next to each 
other. All later stages of the compilation can very easily handle this, 
it's just a slightly different order of inserting stuff into scopes and 
analyzing it.
Jun 01 2022
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 1 June 2022 at 13:48:48 UTC, Timon Gehr wrote:

 There is no need at all to solve this with function pointers 
 the way you describe, just allow forward references and 
 overloading between local functions that are declared next to 
 each other.


The most sensible is to let local functions be syntax sugar for 
lambdas. Just make delegate variables assign-once and 
initialize-before-use.

The core D semantics can be made quite simple with a bit of 
effort. I hope SDC thinks about how to boil the D semantics down 
to a bare minimum…
Jun 01 2022
parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 1 June 2022 at 14:08:24 UTC, Ola Fosheim Grøstad 
wrote:

 The most sensible is to let local functions be syntax sugar for 
 lambdas. Just make delegate variables assign-once and 
 initialize-before-use.


I thought so until I implemented them in SDC, and it actually 
cause many problems. It's not fresh in my mind, so I'm not sure I 
can make a good case for this at the moment, but there were 
definitively problems.

That being said, reducing the semantic discrepancy between the 
two would definitively be a plus.
Jun 02 2022
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 2 June 2022 at 15:57:36 UTC, deadalnix wrote:

 That being said, reducing the semantic discrepancy between the 
 two would definitively be a plus.


I hope it can be resolved somehow, getting to some clean core 
language/high-level-IR can make interprocedural analysis 
realistic (suggested goal: do whatever it takes to make writing 
static analysis fun!).
Jun 02 2022
parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 2 June 2022 at 16:45:29 UTC, Ola Fosheim Grøstad 
wrote:

 On Thursday, 2 June 2022 at 15:57:36 UTC, deadalnix wrote:

 That being said, reducing the semantic discrepancy between the 
 two would definitively be a plus.


 I hope it can be resolved somehow, getting to some clean core 
 language/high-level-IR can make interprocedural analysis 
 realistic (suggested goal: do whatever it takes to make writing 
 static analysis fun!).


I would like to know what you think about sea of node vs SSA on 
this front?

I'm looking at significantly changing SDC's IR now that I have a 
better idea of what it should look like, but I'm not quite sure 
which of the two would be more suitable.
Jun 02 2022
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 2 June 2022 at 21:47:06 UTC, deadalnix wrote:

 I would like to know what you think about sea of node vs SSA on 
 this front?

 I'm looking at significantly changing SDC's IR now that I have 
 a better idea of what it should look like, but I'm not quite 
 sure which of the two would be more suitable.


It is very difficult to be sure about anything a priori as it 
will have to be designed for the passes you want to make, or 
maybe it should be designed so that it is easy to translate to a 
specialized IRs, or maybe it should be designed so that your IR 
can be annotated and extended for future passes that has not been 
decided upon yet. Many options…

It is important to retain all the type information and 
constraints so that they can be exploited. One weakness in Clang 
is that the translation to the LLVM IR has tossed away too much 
information (type/constraints). Maybe ask yourself if there is 
some information worth retaining from template expansion, like 
template constraints? Constraints = opportunities to deduce.

I guess one would want to start by listing the passes you wish to 
perform and the information you need to do so. Maybe look at what 
data structures Swift has landed on to support ARC passes? 
Another thing worth looking at is Boogie, which performs some 
transforms so that asserts in functions can be resolved (on a 
lucky day) by an SMT solver.

I wouldn't say you have to do this or that, but a graph of nodes 
is always the natural option when you want to allow future 
extensions. You can worry about performance later, I would go for 
something that is flexible and "bug resistant". Basic blocks can 
be a single node, if you want, so it might not be an either-or 
choice. I don't think you will be sure what is right until you 
write some passes.

Maybe start with something minimal (for a language subset) and 
try to write som passes for that minimal subset, then you get a 
feel for what you need. I think I would do that, so you can try 
different options with lower costs.
Jun 02 2022
parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 2 June 2022 at 22:36:55 UTC, Ola Fosheim Grøstad 
wrote:

 It is important to retain all the type information and 
 constraints so that they can be exploited. One weakness in 
 Clang is that the translation to the LLVM IR has tossed away 
 too much information (type/constraints). Maybe ask yourself if 
 there is some information worth retaining from template 
 expansion, like template constraints? Constraints = 
 opportunities to deduce.


You'll note that LLVM supports keeping that information around, 
and rust does use this, but C++'s semantic make it effectively 
useless for optimization anyways.


 I guess one would want to start by listing the passes you wish 
 to perform and the information you need to do so. Maybe look at 
 what data structures Swift has landed on to support ARC passes? 
 Another thing worth looking at is Boogie, which performs some 
 transforms so that asserts in functions can be resolved (on a 
 lucky day) by an SMT solver.


I mean, the usual suspect: Combine, CSE, GVN, DCE, simplify CFG, 
Inlining, etc...

As for an SMT solver, that sound like a good way to destroy your 
compile time.

BTW, Swift is using an SSA, but without phi nodes, then instead 
have basic block take arguments and branch instruction take 
parameters.


 Maybe start with something minimal (for a language subset) and 
 try to write som passes for that minimal subset, then you get a 
 feel for what you need. I think I would do that, so you can try 
 different options with lower costs.


SDC already has a full mid level IR that supports most of the 
language.
Jun 02 2022
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 2 June 2022 at 22:50:30 UTC, deadalnix wrote:

 As for an SMT solver, that sound like a good way to destroy 
 your compile time.


Yes, but you can cache results and compile in the background 
while editing and also add an option for distributed analysis 
(cloud solution). Most function bodies dont change frequently. 
All you want to cache is a list of asserts that have been proven 
and what that proof depends on. Besides, this is just a release 
optimization...


 BTW, Swift is using an SSA, but without phi nodes, then instead 
 have basic block take arguments and branch instruction take 
 parameters.


Maybe tell the Swift people that you are considering something 
similar and ask them if they feel that the Swift IR is flexible 
enough, basically ask about the limitations they have seen.


 SDC already has a full mid level IR that supports most of the 
 language.


So, if that is simple enough after template expansion, then that 
would be sufficient for Boogie. Then maybe Swift is not a bad 
starting point if you can get some info from the Swift people.
Jun 02 2022
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 3 June 2022 at 06:42:29 UTC, Ola Fosheim Grøstad wrote:

 On Thursday, 2 June 2022 at 22:50:30 UTC, deadalnix wrote:

 As for an SMT solver, that sound like a good way to destroy 
 your compile time.


 Yes, but you can cache results and compile in the background 
 while editing and also add an option for distributed analysis 
 (cloud solution). Most function bodies dont change frequently. 
 All you want to cache is a list of asserts that have been 
 proven and what that proof depends on. Besides, this is just a 
 release optimization...


Anyway, this shouldn't be a priority, just make sure it is 
possible at a later stage. While it sounds heavy (and is), keep 
in mind that this can be used selectively on bottle necks or 
simple functions with many asserts. And since many D users seem 
to favour releasing with asserts in then this could be that one 
thing that makes SDC faster.

Also if an assert is proven and removed, then it can be assumed 
to hold in LLVM passes.

I guess we also could say that SDC isn't competing to win users 
at this stage, it is competing with DMD, GDC and LDC for the 
attention of D-users interested in compilers.
Jun 03 2022
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Tuesday, 31 May 2022 at 22:48:55 UTC, Don Allen wrote:

 Well, I would argue that that kind of thinking leads to a 
 language that is a collection of special cases, rather than a 
 language built on a core set of principles consistently applied.


While I agree that sometime, D takes the road of many special 
cases, if you take a step back, you'll that this really doesn't 
help your case, to the contrary. You are arguing for special 
casing function scope to allow out of order symbol resolution in 
special circumstances.
Jun 02 2022
parent Don Allen <donaldcallen gmail.com> writes:
On Thursday, 2 June 2022 at 15:15:40 UTC, deadalnix wrote:

 On Tuesday, 31 May 2022 at 22:48:55 UTC, Don Allen wrote:

 Well, I would argue that that kind of thinking leads to a 
 language that is a collection of special cases, rather than a 
 language built on a core set of principles consistently 
 applied.


 While I agree that sometime, D takes the road of many special 
 cases, if you take a step back, you'll that this really doesn't 
 help your case, to the contrary. You are arguing for special 
 casing function scope to allow out of order symbol resolution 
 in special circumstances.


I said I was done, but this really needs a response because you 
have gotten what I am talking about completely wrong, 
particularly in your last sentence above.

I repeat: I am arguing for consistent treatment of the ability to 
do mutual recursion, regardless of context. If D handled this the 
C way -- functions must be defined before they can be referenced 
in another function definition -- and did so in all contexts, I'd 
be less troubled than I am by the current situation, in which we 
have one treatment at module and structure level and the opposite 
within functions.

I could not care less about "helping my case"; my situation 
involves one inner function calling another, so the functions 
aren't even mutually recursive, and is easily resolved by just 
interchanging the order in which the functions are defined. My 
purpose in writing about this has never been about how to fix my 
little issue. It's concern about what I consider an error in the 
language design.
Jun 02 2022
prev sibling next sibling parent Paul Backus <snarwin gmail.com> writes:
On Tuesday, 31 May 2022 at 17:41:18 UTC, Don Allen wrote:

 This strikes me as pretty inconsistent behavior, especially to 
 someone who has written as much Scheme and Haskell as I have. 
 I've also not found it documented, though I could have missed 
 it (if someone could point me to where this is discussed, I'd 
 appreciate it). My main purpose in sending this message is to 
 understand whether this is a bug or a documented "feature". If 
 the former, I will file a bug report.


This behavior (along with several workarounds) is documented in 
the spec here:

https://dlang.org/spec/function.html#nested-declaration-order
May 31 2022
prev sibling next sibling parent "H. S. Teoh" <hsteoh qfbox.info> writes:
On Tue, May 31, 2022 at 05:41:18PM +0000, Don Allen via Digitalmars-d wrote:

 This
 
 ````
 import std.stdio;
 
 void foo() {
     bar();
 }
 void bar() {
     writeln("Hello world");
 }
 int main(string[] args)
 {
     foo();
     return 0;
 }
 ````
 
 compiles and does what you'd expect, despite the forward reference from foo
 to bar, which, of course, would not work in C.
 
 But
 ````
 import std.stdio;
 
 int main(string[] args)
 {
     void foo() {
         bar();
     }
     void bar() {
         writeln("Hello world");
     }
     foo();
     return 0;
 }
 ````
 gives the error
 ````
 (dmd-2.100.0)dca pangloss.allen.net:/home/dca/Software/d_tests$ dmd test5.d
 test5.d(6): Error: undefined identifier `bar`
 ````
 This only works if you interchange the order of foo and bar,
 eliminating the forward reference.
 
 This strikes me as pretty inconsistent behavior, especially to someone
 who has written as much Scheme and Haskell as I have. I've also not
 found it documented, though I could have missed it (if someone could
 point me to where this is discussed, I'd appreciate it). My main
 purpose in sending this message is to understand whether this is a bug
 or a documented "feature". If the former, I will file a bug report.


[...]

This is expected.  In module scope, D tries as much as possible to
eliminate forward reference issues.  (There are some buggy/inconsistent
cases, but those are generally rare unless you're doing something
unusual.)

Inside function scope, however, it was deemed impractical because of
closure over local variables:

	void myfunc() {
		int x;
		void woohoo() {
			x++; // this is legal, closes over x
		}
		...
	}

Had out-of-order declaration been allowed, it would have been rather
confusing:

	void myfunc() {
		void woohoo() {
			x++; // should this close over 'x' inside the 'if'?
		}

		if (someWeirdCondition) {
			int x;
		}
	}

This potentially becomes worse when imported symbols are involved.

Hence, it was decided that inside function scope you must declare all
symbols before referring to them, unlike module scope.


T

-- 
LINUX = Lousy Interface for Nefarious Unix Xenophobes.
May 31 2022
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 31 May 2022 at 17:41:18 UTC, Don Allen wrote:

 
 ````
 import std.stdio;

 int main(string[] args)
 {
     void foo() {
         bar();
     }
     void bar() {
         writeln("Hello world");
     }
     foo();
     return 0;
 }
 ````


This is because you did it in a function body.
Within a function body each declaration opens a scope implicitly.
which gets closed at the end of the parent scope.
This makes sure you don't use variables before they got 
initialized.
```D
int f()
{
// {
   int x;
//   {
   int y = x;  // works
//     {
   int y2 = z;  // does not work because z not in scope
                 // also the value of z isn't defined yet
//       {
   int z;
   // } } } }
}
```
function declarations within a function body follow the same 
rules as variable declarations.
namely a declaration cannot forward reference another in the same 
function body.
May 31 2022
prev sibling next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 31.05.22 19:41, Don Allen wrote:

 
 But
 ````
 import std.stdio;
 
 int main(string[] args)
 {
      void foo() {
          bar();
      }
      void bar() {
          writeln("Hello world");
      }
      foo();
      return 0;
 }
 ````
 gives the error
 ````
 (dmd-2.100.0)dca pangloss.allen.net:/home/dca/Software/d_tests$ dmd test5.d
 test5.d(6): Error: undefined identifier `bar`
 ````
 This only works if you interchange the order of foo and bar, eliminating 
 the
 forward reference.
 
 This strikes me as pretty inconsistent behavior


Not sure if it's necessarily "inconsistent", but it is quite annoying 
and not really necessary. I have also complained about this before.

The simplest workaround is to make `foo` a template:

```d
import std.stdio;

int main(string[] args){
     void foo()(){
         bar();
     }
     void bar(){
         writeln("Hello world");
     }
     foo();
     return 0;
}
```
May 31 2022
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 5/31/22 1:41 PM, Don Allen wrote:

 But
 ````
 import std.stdio;
 
 int main(string[] args)
 {
      void foo() {
          bar();
      }
      void bar() {
          writeln("Hello world");
      }
      foo();
      return 0;
 }
 ````
 gives the error
 ````
 (dmd-2.100.0)dca pangloss.allen.net:/home/dca/Software/d_tests$ dmd test5.d
 test5.d(6): Error: undefined identifier `bar`
 ````
 This only works if you interchange the order of foo and bar, eliminating 
 the
 forward reference.


A solution that comes from a post monkyyy posted about regarding 
function overloading inside a function (also not allowed): 
https://forum.dlang.org/post/uxycfvvbcogmbqrabghk forum.dlang.org

```d
import std.stdio;

int main(string[] args)
{
     template foo() {
         void foo() {
             bar();
         }
         void bar() {
             writeln("Hello world");
         }
     }
     foo();
     return 0;
}
```

As long as you have one entry point to the set of functions, it works as 
expected. If you need to call `bar` directly also, then you need to 
change how the template works, and it's not any nicer than a struct. I 
kind of like this solution, because it doesn't require making types, and 
you don't have to call it differently.

-Steve
Jun 22 2022
parent Paul Backus <snarwin gmail.com> writes:
On Wednesday, 22 June 2022 at 13:05:11 UTC, Steven Schveighoffer 
wrote:

 A solution that comes from a post monkyyy posted about 
 regarding function overloading inside a function (also not 
 allowed): 
 https://forum.dlang.org/post/uxycfvvbcogmbqrabghk forum.dlang.org


If you mixin the template you can call whatever functions you 
want:

```d
import std.stdio;

void main()
{
     template funcs()
     {
         void foo() { bar(); }
         void bar() { writeln("Hello world"); }
     }
     mixin funcs;
     foo();
     bar();
}
```
Jun 22 2022