www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - What does 'inline' mean?

reply Manu <turkeyman gmail.com> writes:
Inline has been bugging me forever, it's usually not what I want. The spec
says this-or-that, but I think we should take a step back, ignore what's
written there, look at the problem space, determine the set of things that
we want, and then make sure they're expressed appropriately.

I think a first part of the conversation to understand, is that since D
doesn't really have first-class `inline` (just a pragma, assumed to be
low-level compiler control), I think most people bring their conceptual
definition over from C/C++, and that definition is a little odd (although
it is immensely useful), but it's not like what D does.

In C/C++, inline says that a function will be emit to the binary only when
it is called, and the function is marked with internal linkage (it is not
visible to the linker from the symbol table)
By this definition; what inline REALLY means is that the function is not
placed in the binary where it is defined, it is placed in the binary where
it is CALLED, and each CU that calls an inline function receives their own
copy of the function. From here; optimisers will typically inline the call
if they determine it's an advantage to do so.

Another take on inline, and perhaps a more natural take (if your mind is
not poisoned by other native languages), is that the function is not
actually emit to an object anywhere, it is rather wired directly inline
into the AST instead of 'called'. Essentially a form of AST macro.

I reach for inline in C/C++ for various different reasons at different
times, and I'd like it if we were able to express each of them:

1. I only want the function to be present in the CALLING binary. I do not
want an inline function present in the local binary where it was defined
(unless it was called internally). I do not want a linker to see the inline
function symbols and be able to link to them externally. [This is about
linkage and controlling the binary or distribution environment]

2. I am unhappy that the optimiser chose to not inline a function call, and
I want to override that judgement. [This is about micro-optimisation]

3. I want to treat the function like an AST macro; I want the function
inserted at the callsite, and I want to have total confidence in this
mechanic. [This is about articulate mechanical control over code-gen; ie, I
know necessary facts about the execution context/callstack that I expect to
maintain]

I think these are the 3 broad categories of behaviour I have ever wanted
control over.
Personally speaking, I am perfectly happy with C/C++'s choice to conflate 1
& 2 into a thing called `inline`, and that's what I want the word 'inline'
to do at an absolute minimum.
The 3rd thing I would term something like `force inline` or perhaps
pragma(inline, force).

D's inline today doesn't do any of those things. It doesn't implement a
mechanic that I have ever wanted or known a use for.

Are there non-theoretical use cases I've missed that people have
encountered?
Jun 07 2020
next sibling parent reply evilrat <evilrat666 gmail.com> writes:
On Monday, 8 June 2020 at 06:14:44 UTC, Manu wrote:

 I think a first part of the conversation to understand, is that 
 since D doesn't really have first-class `inline` (just a 
 pragma, assumed to be low-level compiler control), I think most 
 people bring their conceptual definition over from C/C++, and 
 that definition is a little odd (although it is immensely 
 useful), but it's not like what D does.

 In C/C++, inline says that a function will be emit to the 
 binary only when
 it is called, and the function is marked with internal linkage 
 (it is not
 visible to the linker from the symbol table)
 By this definition; what inline REALLY means is that the 
 function is not
 placed in the binary where it is defined, it is placed in the 
 binary where
 it is CALLED, and each CU that calls an inline function 
 receives their own
 copy of the function. From here; optimisers will typically 
 inline the call
 if they determine it's an advantage to do so.


Sorry for sticking in my nose, but in result inline in C++ simply 
tells the linker "this symbol could be present multiple times, 
just pick whichever one of them and done with it" so it won't 
complain about multiple symbols anymore.
And that's why you have to slap inline to functions that have 
body in header in order to build.

But I'm not very certain if that's all about it, not using C++ 
that much since then, and never to that truly deep extent where 
any loose bit can destroy performance, not beyond fixing the 
client's code and some crappy pet projects.
Jun 08 2020
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/8/2020 1:42 AM, evilrat wrote:

 Sorry for sticking in my nose, but in result inline in C++ simply tells the 
 linker "this symbol could be present multiple times, just pick whichever one
of 
 them and done with it" so it won't complain about multiple symbols anymore.
 And that's why you have to slap inline to functions that have body in header
in 
 order to build.


You're right that the linker expected to combine multiple copies into one.

A more advanced linker will remove unreferenced functions.
Jun 08 2020
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 6/8/20 4:42 AM, evilrat wrote:

 On Monday, 8 June 2020 at 06:14:44 UTC, Manu wrote:

 I think a first part of the conversation to understand, is that since 
 D doesn't really have first-class `inline` (just a pragma, assumed to 
 be low-level compiler control), I think most people bring their 
 conceptual definition over from C/C++, and that definition is a little 
 odd (although it is immensely useful), but it's not like what D does.

 In C/C++, inline says that a function will be emit to the binary only 
 when
 it is called, and the function is marked with internal linkage (it is not
 visible to the linker from the symbol table)
 By this definition; what inline REALLY means is that the function is not
 placed in the binary where it is defined, it is placed in the binary 
 where
 it is CALLED, and each CU that calls an inline function receives their 
 own
 copy of the function. From here; optimisers will typically inline the 
 call
 if they determine it's an advantage to do so.


 
 Sorry for sticking in my nose, but in result inline in C++ simply tells 
 the linker "this symbol could be present multiple times, just pick 
 whichever one of them and done with it" so it won't complain about 
 multiple symbols anymore.
 And that's why you have to slap inline to functions that have body in 
 header in order to build.
 
 But I'm not very certain if that's all about it, not using C++ that much 
 since then, and never to that truly deep extent where any loose bit can 
 destroy performance, not beyond fixing the client's code and some crappy 
 pet projects.


This was not true the last time I found "inline abuse".

At a previous company, there was a type where a function was inline or 
not depending on whether you defined a macro. And when you defined that 
macro, the implementation was DIFFERENT (and different in a way that 
calling the non-inline version would be a bug).

Which meant that if the linker was free to choose whichever 
implementation it wanted, it would defeat the purpose of the inline 
mechanism (BTW, this was the first time I ever experienced an inline 
function, and had to research what it meant).

I think Manu's description is accurate, but I also thought it actually 
inlined the function call as well, even without an optimizer. Hard to 
tell when the end result isn't obvious.

-Steve
Jun 08 2020
next sibling parent reply Manu <turkeyman gmail.com> writes:
On Tue, Jun 9, 2020 at 1:00 AM Steven Schveighoffer via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 6/8/20 4:42 AM, evilrat wrote:

 On Monday, 8 June 2020 at 06:14:44 UTC, Manu wrote:

 I think a first part of the conversation to understand, is that since
 D doesn't really have first-class `inline` (just a pragma, assumed to
 be low-level compiler control), I think most people bring their
 conceptual definition over from C/C++, and that definition is a little
 odd (although it is immensely useful), but it's not like what D does.

 In C/C++, inline says that a function will be emit to the binary only
 when
 it is called, and the function is marked with internal linkage (it is




 not

 visible to the linker from the symbol table)
 By this definition; what inline REALLY means is that the function is not
 placed in the binary where it is defined, it is placed in the binary
 where
 it is CALLED, and each CU that calls an inline function receives their
 own
 copy of the function. From here; optimisers will typically inline the
 call
 if they determine it's an advantage to do so.


 Sorry for sticking in my nose, but in result inline in C++ simply tells
 the linker "this symbol could be present multiple times, just pick
 whichever one of them and done with it" so it won't complain about
 multiple symbols anymore.
 And that's why you have to slap inline to functions that have body in
 header in order to build.

 But I'm not very certain if that's all about it, not using C++ that much
 since then, and never to that truly deep extent where any loose bit can
 destroy performance, not beyond fixing the client's code and some crappy
 pet projects.


 This was not true the last time I found "inline abuse".

 At a previous company, there was a type where a function was inline or
 not depending on whether you defined a macro. And when you defined that
 macro, the implementation was DIFFERENT (and different in a way that
 calling the non-inline version would be a bug).

 Which meant that if the linker was free to choose whichever
 implementation it wanted, it would defeat the purpose of the inline
 mechanism (BTW, this was the first time I ever experienced an inline
 function, and had to research what it meant).

 I think Manu's description is accurate, but I also thought it actually
 inlined the function call as well, even without an optimizer. Hard to
 tell when the end result isn't obvious.


What's funny is, in most cases, whether the function is ACTUALLY inlined is
not really interesting in 2020.
What inline allows is control over the binary environment as I describe. I
read it these days as "inline to the calling CU" rather than "inline to the
calling function".

There are cases where inline is really important, and I do want an error if
it fails; for instance, if you have a leaf function (does not allocate any
stack memory), it's only possible to make calls from that function where
the callee is inlined... and if inlining fails, your caller will lose its
no-stack-frame requirement. I've had this come up numerous times, and in
those cases, a really-strong-does-make-compile-error inline would be
useful, but C++ doesn't have anything like that.
Jun 08 2020
next sibling parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Tuesday, 9 June 2020 at 00:36:18 UTC, Manu wrote:


 What's funny is, in most cases, whether the function is 
 ACTUALLY inlined is
 not really interesting in 2020.
 What inline allows is control over the binary environment as I 
 describe. I
 read it these days as "inline to the calling CU" rather than 
 "inline to the
 calling function".

 There are cases where inline is really important, and I do want 
 an error if it fails; for instance, if you have a leaf function 
 (does not allocate any stack memory), it's only possible to 
 make calls from that function where the callee is inlined... 
 and if inlining fails, your caller will lose its no-stack-frame 
 requirement. I've had this come up numerous times, and in those 
 cases, a really-strong-does-make-compile-error inline would be 
 useful, but C++ doesn't have anything like that.


Maybe it's a case where a clear disambiguation is in order? E.g. 
make a new

pragma(local);

...which would instruct the compiler to do what you're 
describing. Or, perhaps, expand the range of options for the 
existing pragma(inline), from the current bool to an enum of 
behaviors.
Jun 08 2020
parent reply Manu <turkeyman gmail.com> writes:
On Tue, Jun 9, 2020 at 11:30 AM Stanislav Blinov via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Tuesday, 9 June 2020 at 00:36:18 UTC, Manu wrote:


 What's funny is, in most cases, whether the function is
 ACTUALLY inlined is
 not really interesting in 2020.
 What inline allows is control over the binary environment as I
 describe. I
 read it these days as "inline to the calling CU" rather than
 "inline to the
 calling function".

 There are cases where inline is really important, and I do want
 an error if it fails; for instance, if you have a leaf function
 (does not allocate any stack memory), it's only possible to
 make calls from that function where the callee is inlined...
 and if inlining fails, your caller will lose its no-stack-frame
 requirement. I've had this come up numerous times, and in those
 cases, a really-strong-does-make-compile-error inline would be
 useful, but C++ doesn't have anything like that.


 Maybe it's a case where a clear disambiguation is in order? E.g.
 make a new

 pragma(local);

 ...which would instruct the compiler to do what you're
 describing. Or, perhaps, expand the range of options for the
 existing pragma(inline), from the current bool to an enum of
 behaviors.


I've suggested that before, and I think that's what I'd encourage:
  pragma(inline, never) do not inline
  pragma(inline, true) like C/C++, emit to calling CU, hint preference the
optimiser (if it is capable of receiving hints)
  pragma(inline, force) same as true, but error when it fails
Jun 08 2020
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 09/06/2020 1:40 PM, Manu wrote:

 I've suggested that before, and I think that's what I'd encourage:
    pragma(inline, never) do not inline
    pragma(inline, true) like C/C++, emit to calling CU, hint preference 
 the optimiser (if it is capable of receiving hints)
    pragma(inline, force) same as true, but error when it fails


+1
Jun 08 2020
prev sibling next sibling parent Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Tuesday, 9 June 2020 at 00:36:18 UTC, Manu wrote:

 On Tue, Jun 9, 2020 at 1:00 AM Steven Schveighoffer via 
 Digitalmars-d < digitalmars-d puremagic.com> wrote:


[...]


 What's funny is, in most cases, whether the function is 
 ACTUALLY inlined is
 not really interesting in 2020.
 What inline allows is control over the binary environment as I 
 describe. I
 read it these days as "inline to the calling CU" rather than 
 "inline to the
 calling function".

 [...]


Even when forcing inlining with __attribute__((always_inline)) ? 
(in gcc).
Jun 08 2020
prev sibling parent reply Dukc <ajieskola gmail.com> writes:
On Tuesday, 9 June 2020 at 00:36:18 UTC, Manu wrote:

 for instance, if you have a leaf function (does not allocate 
 any stack memory), it's only possible to make calls from that 
 function where the callee is inlined... and if inlining fails, 
 your caller will lose its no-stack-frame requirement.


Out of interest: What does implementing a function without a 
stack frame enable?
Jun 09 2020
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 9 June 2020 at 10:19:44 UTC, Dukc wrote:

 On Tuesday, 9 June 2020 at 00:36:18 UTC, Manu wrote:

 for instance, if you have a leaf function (does not allocate 
 any stack memory), it's only possible to make calls from that 
 function where the callee is inlined... and if inlining fails, 
 your caller will lose its no-stack-frame requirement.


 Out of interest: What does implementing a function without a 
 stack frame enable?


Better predictability of memory access patterns for example.
Knowing that you can't have a stack overflow.
Jun 09 2020
prev sibling parent Manu <turkeyman gmail.com> writes:
On Tue, Jun 9, 2020 at 8:20 PM Dukc via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Tuesday, 9 June 2020 at 00:36:18 UTC, Manu wrote:

 for instance, if you have a leaf function (does not allocate
 any stack memory), it's only possible to make calls from that
 function where the callee is inlined... and if inlining fails,
 your caller will lose its no-stack-frame requirement.


 Out of interest: What does implementing a function without a
 stack frame enable?


Leaf functions have a massive performance advantage, and in certain
situations, functions can run in a zero-ram or zero-stack environment;
micro controllers, within scheduler implementations while performing
operations like context switching, come constructs of hardware exception
handling, etc.
The most common case is maintaining a leaf function's leaf-ness.
Jun 09 2020
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/8/20 10:59 AM, Steven Schveighoffer wrote:

 I think Manu's description is accurate


Most of Manu's description is flat wrong or confuses behavior of 
specific implementations with standard guarantees.
Jun 12 2020
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 6/12/20 9:29 AM, Andrei Alexandrescu wrote:

 On 6/8/20 10:59 AM, Steven Schveighoffer wrote:

 I think Manu's description is accurate


 
 Most of Manu's description is flat wrong or confuses behavior of 
 specific implementations with standard guarantees.


Yeah, I read the inline documentation for C++, and it's quite different 
from what Manu said, and from my experience (this was a long time ago). 
I think the code I had dealt with that used inline was not correct.

-Steve
Jun 12 2020
prev sibling next sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Monday, 8 June 2020 at 06:14:44 UTC, Manu wrote:

 Inline has been bugging me forever, it's usually not what I 
 want. The spec says this-or-that, but I think we should take a 
 step back, ignore what's written there, look at the problem 
 space, determine the set of things that we want, and then make 
 sure they're expressed appropriately.

 I think a first part of the conversation to understand, is that 
 since D doesn't really have first-class `inline` (just a 
 pragma, assumed to be low-level compiler control), I think most 
 people bring their conceptual definition over from C/C++, and 
 that definition is a little odd (although it is immensely 
 useful), but it's not like what D does.

 In C/C++, inline says that a function will be emit to the 
 binary only when
 it is called, and the function is marked with internal linkage 
 (it is not
 visible to the linker from the symbol table)
 By this definition; what inline REALLY means is that the 
 function is not
 placed in the binary where it is defined, it is placed in the 
 binary where
 it is CALLED, and each CU that calls an inline function 
 receives their own
 copy of the function. From here; optimisers will typically 
 inline the call
 if they determine it's an advantage to do so.

 Another take on inline, and perhaps a more natural take (if 
 your mind is not poisoned by other native languages), is that 
 the function is not actually emit to an object anywhere, it is 
 rather wired directly inline into the AST instead of 'called'. 
 Essentially a form of AST macro.


No. I rather see "inline" as a hint for the backend.
DMD is peculiar with its way of inlining.


 I reach for inline in C/C++ for various different reasons at 
 different times, and I'd like it if we were able to express 
 each of them:

 1. I only want the function to be present in the CALLING 
 binary. I do not want an inline function present in the local 
 binary where it was defined (unless it was called internally). 
 I do not want a linker to see the inline function symbols and 
 be able to link to them externally. [This is about linkage and 
 controlling the binary or distribution environment]


what if the function address is took in a delegate ?
It still needs to be there, in the object matching to the CU 
where it is declared, otherwise there will be surprises, e.g 
&func in a CU and &func in another will have different addresses.
Jun 08 2020
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 8 June 2020 at 08:45:02 UTC, Basile B. wrote:


 what if the function address is took in a delegate ?
 It still needs to be there, in the object matching to the CU 
 where it is declared, otherwise there will be surprises, e.g 
 &func in a CU and &func in another will have different 
 addresses.


I have _never_ compared addresses of inline functions from 
different CUs.
How would I even do that?
by storing the function pointer in a global which is visible from 
multiple translation units?
Is that a valid concern?
Will _any_ project out there break because of that?
Jun 08 2020
next sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Monday, 8 June 2020 at 08:47:36 UTC, Stefan Koch wrote:

 On Monday, 8 June 2020 at 08:45:02 UTC, Basile B. wrote:


 what if the function address is took in a delegate ?
 It still needs to be there, in the object matching to the CU 
 where it is declared, otherwise there will be surprises, e.g 
 &func in a CU and &func in another will have different 
 addresses.


 I have _never_ compared addresses of inline functions from 
 different CUs.
 How would I even do that?
 by storing the function pointer in a global which is visible 
 from multiple translation units?
 Is that a valid concern?
 Will _any_ project out there break because of that?


It's a detail. I just meant that in case where the address of 
function that is marked for inlining is took then it must still 
be emitted in the object matching to the declaration unit.
Jun 08 2020
next sibling parent Basile B. <b2.temp gmx.com> writes:
On Monday, 8 June 2020 at 09:08:33 UTC, Basile B. wrote:

 On Monday, 8 June 2020 at 08:47:36 UTC, Stefan Koch wrote:

 On Monday, 8 June 2020 at 08:45:02 UTC, Basile B. wrote:


 [...]


 I have _never_ compared addresses of inline functions from 
 different CUs.
 How would I even do that?
 by storing the function pointer in a global which is visible 
 from multiple translation units?
 Is that a valid concern?
 Will _any_ project out there break because of that?


 It's a detail. I just meant that in case where the address of 
 function that is marked for inlining is took then it must still 
 be emitted in the object matching to the declaration unit.


but of course inlined where it is used as-is.
Jun 08 2020
prev sibling parent Manu <turkeyman gmail.com> writes:
On Mon, Jun 8, 2020 at 7:10 PM Basile B. via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Monday, 8 June 2020 at 08:47:36 UTC, Stefan Koch wrote:

 On Monday, 8 June 2020 at 08:45:02 UTC, Basile B. wrote:


 what if the function address is took in a delegate ?
 It still needs to be there, in the object matching to the CU
 where it is declared, otherwise there will be surprises, e.g
 &func in a CU and &func in another will have different
 addresses.


 I have _never_ compared addresses of inline functions from
 different CUs.
 How would I even do that?
 by storing the function pointer in a global which is visible
 from multiple translation units?
 Is that a valid concern?
 Will _any_ project out there break because of that?


 It's a detail. I just meant that in case where the address of
 function that is marked for inlining is took then it must still
 be emitted in the object matching to the declaration unit.


It should be emit to the CU where the address is taken.
Jun 08 2020
prev sibling parent reply Dukc <ajieskola gmail.com> writes:
On Monday, 8 June 2020 at 08:47:36 UTC, Stefan Koch wrote:

 Is that a valid concern?
 Will _any_ project out there break because of that?


It's easy to imagine. If one has a very simple mathematical 
function taking, say int and float and returning a float, and 
that function is called in a very hot loop, performance can drop 
noticeably if inlining suddently fails. Especially if the 
mathematical function calls another function instead of using the 
compiler primitives directly.
Jun 08 2020
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 8 June 2020 at 11:53:11 UTC, Dukc wrote:

 On Monday, 8 June 2020 at 08:47:36 UTC, Stefan Koch wrote:

 Is that a valid concern?
 Will _any_ project out there break because of that?


 It's easy to imagine. If one has a very simple mathematical 
 function taking, say int and float and returning a float, and 
 that function is called in a very hot loop, performance can 
 drop noticeably if inlining suddently fails. Especially if the 
 mathematical function calls another function instead of using 
 the compiler primitives directly.


I didn't ask about not inlinng.
I asked about the address of a function changing.
 From object file to object file
Jun 08 2020
prev sibling parent Manu <turkeyman gmail.com> writes:
On Mon, Jun 8, 2020 at 6:50 PM Basile B. via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Monday, 8 June 2020 at 06:14:44 UTC, Manu wrote:

 Inline has been bugging me forever, it's usually not what I
 want. The spec says this-or-that, but I think we should take a
 step back, ignore what's written there, look at the problem
 space, determine the set of things that we want, and then make
 sure they're expressed appropriately.

 I think a first part of the conversation to understand, is that
 since D doesn't really have first-class `inline` (just a
 pragma, assumed to be low-level compiler control), I think most
 people bring their conceptual definition over from C/C++, and
 that definition is a little odd (although it is immensely
 useful), but it's not like what D does.

 In C/C++, inline says that a function will be emit to the
 binary only when
 it is called, and the function is marked with internal linkage
 (it is not
 visible to the linker from the symbol table)
 By this definition; what inline REALLY means is that the
 function is not
 placed in the binary where it is defined, it is placed in the
 binary where
 it is CALLED, and each CU that calls an inline function
 receives their own
 copy of the function. From here; optimisers will typically
 inline the call
 if they determine it's an advantage to do so.

 Another take on inline, and perhaps a more natural take (if
 your mind is not poisoned by other native languages), is that
 the function is not actually emit to an object anywhere, it is
 rather wired directly inline into the AST instead of 'called'.
 Essentially a form of AST macro.


 No. I rather see "inline" as a hint for the backend.
 DMD is peculiar with its way of inlining.


 I reach for inline in C/C++ for various different reasons at
 different times, and I'd like it if we were able to express
 each of them:

 1. I only want the function to be present in the CALLING
 binary. I do not want an inline function present in the local
 binary where it was defined (unless it was called internally).
 I do not want a linker to see the inline function symbols and
 be able to link to them externally. [This is about linkage and
 controlling the binary or distribution environment]


 what if the function address is took in a delegate ?
 It still needs to be there, in the object matching to the CU
 where it is declared, otherwise there will be surprises, e.g
 &func in a CU and &func in another will have different addresses.



Right. This is fine and normal.
Jun 08 2020
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/7/2020 11:14 PM, Manu wrote:

 I think a first part of the conversation to understand, is that since D
doesn't 
 really have first-class `inline` (just a pragma, assumed to be low-level 
 compiler control), I think most people bring their conceptual definition over 
 from C/C++, and that definition is a little odd (although it is immensely 
 useful), but it's not like what D does.


C/C++ inline has always been a hint to the compiler, not a command.



 In C/C++, inline says that a function will be emit to the binary only when it
is 
 called, and the function is marked with internal linkage (it is not visible to 
 the linker from the symbol table)
 By this definition; what inline REALLY means is that the function is not
placed 
 in the binary where it is defined, it is placed in the binary where it is 
 CALLED, and each CU that calls an inline function receives their own copy of
the 
 function.


Why does it matter where it is emitted? Why would you want multiple copies of 
the same function in the binary?



 Another take on inline, and perhaps a more natural take (if your mind is not 
 poisoned by other native languages), is that the function is not actually emit 
 to an object anywhere, it is rather wired directly inline into the AST instead 
 of 'called'. Essentially a form of AST macro.


The problem with this is what is inlined and what isn't is rather fluid, i.e.
it 
varies depending on circumstances and which compiler you use. For example, if 
you recursively call a function, it's going to have to give up on inlining it. 
Changes in the compiler can expand or contract inlining opportunities. Having
it 
inline or issue an error is disaster for compiling existing code without 
constantly breaking it.



 I reach for inline in C/C++ for various different reasons at different times, 
 and I'd like it if we were able to express each of them:
 
 1. I only want the function to be present in the CALLING binary. I do not want 
 an inline function present in the local binary where it was defined (unless it 
 was called internally). I do not want a linker to see the inline function 
 symbols and be able to link to them externally. [This is about linkage and 
 controlling the binary or distribution environment]


Why? What is the problem with the emission of one copy where it was defined?



 2. I am unhappy that the optimiser chose to not inline a function call, and I 
 want to override that judgement. [This is about micro-optimisation]


It's not always possible to inline a function.



 3. I want to treat the function like an AST macro; I want the function
inserted 
 at the callsite, and I want to have total confidence in this mechanic. [This
is 
 about articulate mechanical control over code-gen; ie, I know necessary facts 
 about the execution context/callstack that I expect to maintain]


The PR I have on this makes it an informational warning. You can choose to be 
notified if inlining fails.



 Are there non-theoretical use cases I've missed that people have encountered?


At its root, inlining is an optimization, like deciding which variables go into 
registers.
Jun 08 2020
next sibling parent reply Jan =?UTF-8?B?SMO2bmln?= <hrominium gmail.com> writes:
On Monday, 8 June 2020 at 10:19:16 UTC, Walter Bright wrote:

 Why does it matter where it is emitted? Why would you want 
 multiple copies of the same function in the binary?


Performance in HPC.

In C++, consider an []operator. There would be a lot of function 
calls inside a kernel (some function with lot of loops, one 
billion iterations of the inner most loop easily). If then I have 
some kind of stencil or any array accesses, calling a function 
each time a top of resolving the current pointer would be very 
costly.
Jun 08 2020
next sibling parent John Colvin <john.loughran.colvin gmail.com> writes:
On Monday, 8 June 2020 at 12:35:01 UTC, Jan Hönig wrote:

 On Monday, 8 June 2020 at 10:19:16 UTC, Walter Bright wrote:

 Why does it matter where it is emitted? Why would you want 
 multiple copies of the same function in the binary?


 Performance in HPC.

 In C++, consider an []operator. There would be a lot of 
 function calls inside a kernel (some function with lot of 
 loops, one billion iterations of the inner most loop easily). 
 If then I have some kind of stencil or any array accesses, 
 calling a function each time a top of resolving the current 
 pointer would be very costly.


What does this have to do with whether a symbol is emitted? 
Emitting a symbol doesn't mean you didn't get inlining.
Jun 08 2020
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/8/2020 5:35 AM, Jan Hönig wrote:

 In C++, consider an []operator. There would be a lot of function calls inside
a 
 kernel (some function with lot of loops, one billion iterations of the inner 
 most loop easily). If then I have some kind of stencil or any array accesses, 
 calling a function each time a top of resolving the current pointer would be 
 very costly.


I infer what you're talking about is functions that are "strongly connected" 
should be located near each other in memory so they'll both be in the cache at 
the same time.

The best way to achieve this is by runtime profiling, and using the profiling 
data to group together strongly connected function in the executable. The 
Digital Mars C/C++ compiler would do this, and it was a nice optimization.

Trying to do it by hand isn't likely to be very effective.
Jun 09 2020
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On Mon, Jun 8, 2020 at 8:20 PM Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 6/7/2020 11:14 PM, Manu wrote:

 I think a first part of the conversation to understand, is that since D


 doesn't

 really have first-class `inline` (just a pragma, assumed to be low-level
 compiler control), I think most people bring their conceptual definition


 over

 from C/C++, and that definition is a little odd (although it is


 immensely

 useful), but it's not like what D does.


 C/C++ inline has always been a hint to the compiler, not a command.


It's not a hint at all. It's a mechanical tool; it marks symbols with
internal linkage, and it also doesn't emit them if it's never referenced.
The compiler may not choose to ignore that behaviour, it's absolutely
necessary, and very important.


 In C/C++, inline says that a function will be emit to the binary only
 when it is

 called, and the function is marked with internal linkage (it is not


 visible to

 the linker from the symbol table)
 By this definition; what inline REALLY means is that the function is not


 placed

 in the binary where it is defined, it is placed in the binary where it


 is

 CALLED, and each CU that calls an inline function receives their own


 copy of the

 function.


 Why does it matter where it is emitted? Why would you want multiple copies
 of
 the same function in the binary?


I want zero copies if it's never called. That's very important.
I also want copies to appear locally when it is referenced; inline
functions should NOT require that you link something to get the code...
that's not inline at all.


 Another take on inline, and perhaps a more natural take (if your mind is
 not

 poisoned by other native languages), is that the function is not


 actually emit

 to an object anywhere, it is rather wired directly inline into the AST


 instead

 of 'called'. Essentially a form of AST macro.


 The problem with this is what is inlined and what isn't is rather fluid,
 i.e. it
 varies depending on circumstances and which compiler you use. For example,
 if
 you recursively call a function, it's going to have to give up on inlining
 it.
 Changes in the compiler can expand or contract inlining opportunities.
 Having it
 inline or issue an error is disaster for compiling existing code without
 constantly breaking it.


I understand why this particular take is more complicated; and as such I
wouldn't suggest we do it. I'm just offering it as one possible take on the
concept.
I think to make this form work, it must be handled in the frontend;
essentially an AST macro. Naturally, it would fail on recursive calls,
because that would be a recursive expansion.

case could potentially be this.


 I reach for inline in C/C++ for various different reasons at different
 times,

 and I'd like it if we were able to express each of them:

 1. I only want the function to be present in the CALLING binary. I do


 not want

 an inline function present in the local binary where it was defined


 (unless it

 was called internally). I do not want a linker to see the inline


 function

 symbols and be able to link to them externally. [This is about linkage


 and

 controlling the binary or distribution environment]


 Why? What is the problem with the emission of one copy where it was
 defined?


That's the antithesis of inline. If I wanted that, I wouldn't mark it
inline.
I don't want a binary full of code that shouldn't be there. It's very
important to be able to control what code is in your binaries.

If it's not referenced, it doesn't exist.


 2. I am unhappy that the optimiser chose to not inline a function call,
 and I

 want to override that judgement. [This is about micro-optimisation]


 It's not always possible to inline a function.






 3. I want to treat the function like an AST macro; I want the function
 inserted

 at the callsite, and I want to have total confidence in this mechanic.


 [This is

 about articulate mechanical control over code-gen; ie, I know necessary


 facts

 about the execution context/callstack that I expect to maintain]


 The PR I have on this makes it an informational warning. You can choose to
 be
 notified if inlining fails.


That's not sufficient though for all use cases. This is a different kind of
inline (I think it's 'force inline').

want a sea of warnings to apply to cases of 1/2.



 Are there non-theoretical use cases I've missed that people have
 encountered?

 At its root, inlining is an optimization, like deciding which variables go
 into
 registers.


No, actually... it's not. It's not an 'optimisation' in any case except

Low level control of code generation is important in native languages;
that's why we're here.
Jun 08 2020
next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Monday, 8 June 2020 at 14:09:04 UTC, Manu wrote:

 [snip]
 It's not a hint at all. It's a mechanical tool; it marks 
 symbols with internal linkage, and it also doesn't emit them if 
 it's never referenced. The compiler may not choose to ignore 
 that behaviour, it's absolutely necessary, and very important.
[snip]


Perhaps it would be helpful to split the discussion into inline 
as a linkage attribute vs. inline as an tool for inline 
expansion? D has no linkage attribute that has the same behavior 
as inline or extern inline, and you believe that is important.
Jun 08 2020
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/8/2020 7:09 AM, Manu wrote:

 On Mon, Jun 8, 2020 at 8:20 PM Walter Bright via Digitalmars-d 
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>> wrote:
 
     On 6/7/2020 11:14 PM, Manu wrote:
      > I think a first part of the conversation to understand, is that since D
     doesn't
      > really have first-class `inline` (just a pragma, assumed to be low-level
      > compiler control), I think most people bring their conceptual definition
     over
      > from C/C++, and that definition is a little odd (although it is
immensely
      > useful), but it's not like what D does.
 
     C/C++ inline has always been a hint to the compiler, not a command.
 
 
 It's not a hint at all. It's a mechanical tool; it marks symbols with internal 
 linkage, and it also doesn't emit them if it's never referenced.
 The compiler may not choose to ignore that behaviour,


The C/C++ inline semantics revolve around the mechanics of .h files because it 
doesn't have modules. These reasons are irrelevant for D.


 it's absolutely necessary, and very important.


For .h files, sure. Why for D, though?


     Why does it matter where it is emitted? Why would you want multiple copies
of
     the same function in the binary?
 I want zero copies if it's never called. That's very important.


Why are 0 or N copies fine, but 1 is not?



 I also want copies to appear locally when it is referenced; inline functions 
 should NOT require that you link something to get the code... that's not
inline 
 at all.


Why? What problem are you solving?



     Why? What is the problem with the emission of one copy where it was
defined?


 That's the antithesis of inline. If I wanted that, I wouldn't mark it inline.
 I don't want a binary full of code that shouldn't be there. It's very
important 
 to be able to control what code is in your binaries.


I know I'm being boring, but why is it important? Also, check out -gc-sections:

https://gcc.gnu.org/onlinedocs/gnat_ugn/Compilation-options.html

Which is a general solution, not a kludge.



 If it's not referenced, it doesn't exist.


Executables (on virtual memory systems) are not loaded into memory and then
run. 
They are memory-mapped into memory, and then pages are read off of disk on 
demand. Unmapped code consumes neither memory nor resources.



     The PR I have on this makes it an informational warning. You can choose to
be
     notified if inlining fails.


 That's not sufficient though for all use cases. This is a different kind of 
 inline (I think it's 'force inline').


The default, and pragma(inline,true) are sufficient for all use cases except 
which ones?



 sea of warnings to apply to cases of 1/2.


You won't get a sea of warnings unless you put pragma(inline,true) on a sea of 
functions that can't be inlined.





Why?



   At its root, inlining is an optimization, like deciding which variables go
into
   registers.


 No, actually... it's not. It's not an 'optimisation' in any case except
maaaaybe 



Inlining is 100% about optimization.



 Low level control of code generation is important in native languages; that's 
 why we're here.


Optimizing things that don't matter is wasting your valuable time. Optimizing 
things that are more effectively and thoroughly done with the linker 
(-gc-sections) - it's like chipping wood with a hatchet rather than a
woodchipper.
Jun 09 2020
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 9 June 2020 at 09:29:47 UTC, Walter Bright wrote:

 On 6/8/2020 7:09 AM, Manu wrote:

 No, actually... it's not. It's not an 'optimisation' in any 

 output and code generation.


 Inlining is 100% about optimization.


We are not talking about inlining.

The whole idea behind this thread is to explain that, c++ use of 
"inline" does not actually mean that a function is to be inlined.

What we are talking about is a non-hacky way to say, I want a 
copy of this function in my object file, regardless of the module 
it was defined in.

You _can_ get the same effect by doing this:
---
string I_need_this_function()(string x)
{
     return "x: " ~ x;
}
---

But that introduces a template which comes with another set of 
problems.
Such that you don't get type-checking of the body if it's not 
used.
Jun 09 2020
prev sibling next sibling parent Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Tuesday, 9 June 2020 at 09:29:47 UTC, Walter Bright wrote:

 On 6/8/2020 7:09 AM, Manu wrote:

 [...]


 The C/C++ inline semantics revolve around the mechanics of .h 
 files because it doesn't have modules. These reasons are 
 irrelevant for D.


 [...]


 For .h files, sure. Why for D, though?


 [...]


 Why are 0 or N copies fine, but 1 is not?



 [...]


 Why? What problem are you solving?



 [...]


 I know I'm being boring, but why is it important? Also, check 
 out -gc-sections:

 https://gcc.gnu.org/onlinedocs/gnat_ugn/Compilation-options.html

 Which is a general solution, not a kludge.



 [...]


 Executables (on virtual memory systems) are not loaded into 
 memory and then run. They are memory-mapped into memory, and 
 then pages are read off of disk on demand. Unmapped code 
 consumes neither memory nor resources.



 [...]


 The default, and pragma(inline,true) are sufficient for all use 
 cases except which ones?


 [...]


 You won't get a sea of warnings unless you put 
 pragma(inline,true) on a sea of functions that can't be inlined.


 [...]


 Why?



 [...]


 Inlining is 100% about optimization.


You forgot the case where it is about pessimization :-)

The thing with inlining is that it often looks good in benchmarks 
but not that good in whole applications because of the impact it 
can have on code size.
In my modest experience, the worse offenders where always the 
people that insisted very much on forcing inlining. I had once a 
colleague that had put inlines everywhere in his lib without 
realizing that nesting inlined functions has a multiplying 
effect. I calculated the equivalent in line of codes that were 
inlined and it was around 6000, for a library that had only 13000 
lines to begin with.
The effect on the instruction cache was huge.


 [...]


 Optimizing things that don't matter is wasting your valuable 
 time. Optimizing things that are more effectively and 
 thoroughly done with the linker (-gc-sections) - it's like 
 chipping wood with a hatchet rather than a woodchipper.
Jun 09 2020
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On Tue, Jun 9, 2020 at 7:30 PM Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 6/8/2020 7:09 AM, Manu wrote:

 On Mon, Jun 8, 2020 at 8:20 PM Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>>


 wrote:

     On 6/7/2020 11:14 PM, Manu wrote:
      > I think a first part of the conversation to understand, is that


 since D

     doesn't
      > really have first-class `inline` (just a pragma, assumed to be


 low-level

      > compiler control), I think most people bring their conceptual


 definition

     over
      > from C/C++, and that definition is a little odd (although it is


 immensely

      > useful), but it's not like what D does.

     C/C++ inline has always been a hint to the compiler, not a command.


 It's not a hint at all. It's a mechanical tool; it marks symbols with


 internal

 linkage, and it also doesn't emit them if it's never referenced.
 The compiler may not choose to ignore that behaviour,


 The C/C++ inline semantics revolve around the mechanics of .h files
 because it
 doesn't have modules. These reasons are irrelevant for D.


That's completely incorrect. It's 100% as relevant for D as it is for C++
for exactly the same reasons.
You'll need to support your claim.


 it's absolutely necessary, and very important.

 For .h files, sure. Why for D, though?


Because in this category of use case, inlining is a concept related to
native languages with binary linkage, and not really anything to do with
the language specifically.
Calling an inline function from a foreign module does not require that I
link that module's compiled code, because the inline function is emit
locally to the calling CU.
This is just as relevant in D as it is in C++, although the technical
manifestations are slightly different; in C++, a function defined in a
header emits a hard-copy of the function in each CU, and link errors
because multiple defined symbol. In D, you end up with no copy of the
function anywhere, and link errors because undefined symbol. `Inline`
addresses that same issue in both cases the same way.

There are a whole lot of reasons this comes up in binary ecosystems. Build
systems and dev tooling is a really complex topic, which tends to take
years and years of iteration for millions-loc software, and there are
frequently awkward requirements for various reasons. We need to fit in with
existing expectations.
It must be painless to plug into existing native code ecosystems; we have
invested heavily in extern C and C++, but compatibility with those
ecosystems also involves integrating into complex existing legacy build,
link, and distribution environments.


     Why does it matter where it is emitted? Why would you want multiple
 copies of

     the same function in the binary?
 I want zero copies if it's never called. That's very important.


 Why are 0 or N copies fine, but 1 is not?


No, that's not what I've said; I expect exactly N copies of x() for N CU's
which reference x(). And they should appropriately be marked with internal
linkage.
Any other result is just 'weird', and while it might be workable, it's just
asking for trouble. (1 redundant copy in the owning CU despite being
un-referenced MIGHT be link-stripped if the surrounding tooling all works
as we hope... but it also might not, as I have demonstrated on multiple
occasions)
There's just no reason to invite this problem... and no advantage.


 I also want copies to appear locally when it is referenced; inline
 functions

 should NOT require that you link something to get the code... that's not


 inline

 at all.


 Why? What problem are you solving?


Literally inline function calling. No-link libs are a really common and
extremely useful thing.


     Why? What is the problem with the emission of one copy where it was


 defined?

 That's the antithesis of inline. If I wanted that, I wouldn't mark it


 inline.

 I don't want a binary full of code that shouldn't be there. It's very


 important

 to be able to control what code is in your binaries.


 I know I'm being boring, but why is it important? Also, check out
 -gc-sections:

 https://gcc.gnu.org/onlinedocs/gnat_ugn/Compilation-options.html

 Which is a general solution, not a kludge.


I guess my key issue is that I have complained before because I have
experienced multiple counts of the link stripping not working like you say.
There is no reason to invite that problem; we can trivially eliminate the
problem at the source. I don't care if it's fixable, I don't want the
problem to exist. NOBODY wants to be tasked to solve awkward build
ecosystem issues... we already have a working build ecosystem, and this D
thing is making it harder than it already is. That's a really bad thing,
and I would entertain excuses for this if not for the fact that it's
trivially avoidable. We do not need to waste anybodies time this way; we
won't win any friends by wasting their time with problems they HATE to have.

The secondary issue is, I want my binaries to contain what I put in there,
and not what I don't. Rogue symbols that I specified shouldn't exist only
bloat the binary, invite possibility of link collision, and raise the
probability of the issues I mentioned above.


     The PR I have on this makes it an informational warning. You can


 choose to be

     notified if inlining fails.


 That's not sufficient though for all use cases. This is a different kind


 of

 inline (I think it's 'force inline').


 The default, and pragma(inline,true) are sufficient for all use cases
 except
 which ones?


Recursive calls or taking the address of functions (and probably other
situations) are incompatible with a hard-error based inline.
CU-inline is the overwhelmingly common case.
Absolutely-must-call-site-inline is extremely rare by contrast, but very
important in the rare instance it's necessary.

I suggest, the default should model the common case, and the rare niche
case can be the 3rd 'force' state by explicit request.



 want a

 sea of warnings to apply to cases of 1/2.


 You won't get a sea of warnings unless you put pragma(inline,true) on a
 sea of
 functions that can't be inlined.


There are classes of modules where EVERY function is inline. No-link libs
are a very common and useful tool.



 cases.

 Why?





territory, which you need to manually validate, and then have no way to
detect when the conditions or context changes.




along the way...


   At its root, inlining is an optimization, like deciding which


 variables go into

   registers.


 No, actually... it's not. It's not an 'optimisation' in any case except


 maaaaybe




 Inlining is 100% about optimization.


No. I feel like I couldn't make this case clearer... It's got almost
nothing to do with optimisation, it's about codegen control. In the very
rare event that I disagree with an optimisers heuristic, it's nice to have
an override hint, but that's like a 0.1% use case. Inline provides
mechanical control over codegen, and this is a native systems language. We
must have that control.
GDC/LDC have it, but it really should be specced to have uniform behaviour
across D compilers. We don't have macro's like C++ does to wrangle cases
where implementations differ as easily.


 Low level control of code generation is important in native languages;
 that's

 why we're here.


 Optimizing things that don't matter is wasting your valuable time.
 Optimizing
 things that are more effectively and thoroughly done with the linker
 (-gc-sections) - it's like chipping wood with a hatchet rather than a
 woodchipper.


This isn't about optimisation, it's about controlling the output of the
compiler. Taking that control away and forcing us to try and reproduce that
expected functionality with external tooling within a deeply complex build
ecosystem is wasting our valuable time.
Jun 09 2020
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/9/2020 7:21 AM, Manu wrote:

     The C/C++ inline semantics revolve around the mechanics of .h files
because it
     doesn't have modules. These reasons are irrelevant for D.
 
 
 That's completely incorrect. It's 100% as relevant for D as it is for C++ for 
 exactly the same reasons.
 You'll need to support your claim.


Um, because of the way #include works, C++ sees the same inline function 
definition over and over again, and has no way of knowing if any other files
see 
the same definition or not, or if they have the same implementation or not. The 
semantics are carefully crafted around the reality that there may be multiple 
copies of the inline function definition in compilation units that have no clue 
of the existence of each other. The internal linkage thing is there to support 
more primitive linkers that have no support for COMDAT sections.

This is completely different from a module system. Also, D relies heavily on 
linker support for COMDATs.



 Because in this category of use case, inlining is a concept related to native 
 languages with binary linkage, and not really anything to do with the language 
 specifically.


Given all the verbiage about inlines in the C++ spec, it does have to do with 
the language specifically.



 Any other result is just 'weird', and while it might be workable, it's just 
 asking for trouble. (1 redundant copy in the owning CU despite being 
 un-referenced MIGHT be link-stripped if the surrounding tooling all works as
we 
 hope... but it also might not, as I have demonstrated on multiple occasions)
 There's just no reason to invite this problem... and no advantage.
 
      > I also want copies to appear locally when it is referenced; inline
functions
      > should NOT require that you link something to get the code... that's not
     inline
      > at all.
 
     Why? What problem are you solving?
 
 
 Literally inline function calling. No-link libs are a really common and 
 extremely useful thing.


At last! You say why! We have that in D with template functions. You get N 
instantiations. But they all have COMDAT linkage, which means only 1 winds up
in 
the executable instead of N.



     I know I'm being boring, but why is it important? Also, check out
-gc-sections:
 
     https://gcc.gnu.org/onlinedocs/gnat_ugn/Compilation-options.html
 
     Which is a general solution, not a kludge.
 
 
 I guess my key issue is that I have complained before because I have
experienced 
 multiple counts of the link stripping not working like you say. There is no 
 reason to invite that problem; we can trivially eliminate the problem at the 
 source. I don't care if it's fixable, I don't want the problem to exist.
NOBODY 
 wants to be tasked to solve awkward build ecosystem issues... we already have
a 
 working build ecosystem, and this D thing is making it harder than it already 
 is. That's a really bad thing, and I would entertain excuses for this if not
for 
 the fact that it's trivially avoidable. We do not need to waste anybodies time 
 this way; we won't win any friends by wasting their time with problems they
HATE 
 to have.


If gc-sections isn't working, perhaps there's something else going on. Did you 
check with the linker support people? gc-sections has been around at least 20 
years with gcc. If it fundamentally didn't work somebody would have fixed it by
now.

It also may not work with your C++ compiler because it is NOT emitting the 
functions each into its own section. DMD *does* put each and every function
into 
its own section, so that gc-sections will work.

I.e. C++ has workarounds for bugs it its own implementation, and those 
workarounds are not features.



 The secondary issue is, I want my binaries to contain what I put in there, and 
 not what I don't. Rogue symbols that I specified shouldn't exist only bloat
the 
 binary, invite possibility of link collision, and raise the probability of the 
 issues I mentioned above.


But you're happy with (and require) N useless redundant copies of inline
functions.



 I suggest, the default should model the common case, and the rare niche case
can 
 be the 3rd 'force' state by explicit request.


The upcoming case is pragma(inline,true) (in my PR) is that the compiler 
attempts to inline them regardless of the -inline switch, and lists an 
informational warning (-wi) if they can't be inlined.



 This isn't about optimisation, it's about controlling the output of the 
 compiler. Taking that control away and forcing us to try and reproduce that 
 expected functionality with external tooling within a deeply complex build 
 ecosystem is wasting our valuable time.


C++'s inline semantics are rooted in accommodation for:

1. .h files rather than modules
2. object file formats that don't support COMDATs
3. linkers that don't support COMDATs

It's a collection of hacks, not features. D is free'd from those constraints.
Jun 09 2020
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Jun 09, 2020 at 04:48:17PM -0700, Walter Bright via Digitalmars-d wrote:

 On 6/9/2020 7:21 AM, Manu wrote:


[...]

   Why? What problem are you solving?


 
 
 Literally inline function calling. No-link libs are a really common and
 extremely useful thing.


 
 At last! You say why! We have that in D with template functions. You
 get N instantiations. But they all have COMDAT linkage, which means
 only 1 winds up in the executable instead of N.


[...]

Yeah, I was about to say, judging from what Manu has written several
times in this thread, his concept of 'inline' bears striking resemblance
to a template function:

- The template function is not instantiated in its defining module
  (unless something else instantiates it there);

- A template function is instantiated in the invoking module, just like
  how Manu described his 'inline' functions;

- A template function is more likely to be actually inlined (recently
  I've been adding () to some of my functions in order to prod the
  compiler to inline them).  Manu has even stated that whether or not
  the 'inline' function is actually inlined isn't even that important
  anymore; given that, I'd say template functions are what he's looking
  for all this time!


T

-- 
Why ask rhetorical questions? -- JC
Jun 09 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 10 June 2020 at 00:02:09 UTC, H. S. Teoh wrote:

 On Tue, Jun 09, 2020 at 04:48:17PM -0700, Walter Bright via 
 Digitalmars-d wrote:

 [...]


 [...]

 [...]


 [...]

 Yeah, I was about to say, judging from what Manu has written 
 several times in this thread, his concept of 'inline' bears 
 striking resemblance to a template function:

 [...]


Yes making the function a function template will do that.
And Manu is fully aware of it.

It's a nasty hack.
And in certain cases it screws with overload-resolution.
Jun 09 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 6/9/20 8:20 PM, Stefan Koch wrote:

 On Wednesday, 10 June 2020 at 00:02:09 UTC, H. S. Teoh wrote:

 On Tue, Jun 09, 2020 at 04:48:17PM -0700, Walter Bright via 
 Digitalmars-d wrote:

 [...]


 [...]

 [...]


 [...]

 Yeah, I was about to say, judging from what Manu has written several 
 times in this thread, his concept of 'inline' bears striking 
 resemblance to a template function:

 [...]


 
 Yes making the function a function template will do that.
 And Manu is fully aware of it.
 
 It's a nasty hack.
 And in certain cases it screws with overload-resolution.


It's not exactly the same. If the compiler thinks an imported module has 
instantiated it, it will not include it in the object file (unless you 
use -allinst).

-Steve
Jun 10 2020
parent reply kinke <noone nowhere.com> writes:
On Wednesday, 10 June 2020 at 13:55:40 UTC, Steven Schveighoffer 
wrote:

 On 6/9/20 8:20 PM, Stefan Koch wrote:

 Yes making the function a function template will do that.
 And Manu is fully aware of it.
 
 It's a nasty hack.
 And in certain cases it screws with overload-resolution.


 It's not exactly the same. If the compiler thinks an imported 
 module has instantiated it, it will not include it in the 
 object file (unless you use -allinst).


Nope, -allinst doesn't make sure all templates are instantiated 
in each object file. I'll link again to 
https://github.com/ldc-developers/ldc/pull/3422#issuecomment-625945508.
Jun 10 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 6/10/20 2:54 PM, kinke wrote:

 On Wednesday, 10 June 2020 at 13:55:40 UTC, Steven Schveighoffer wrote:

 On 6/9/20 8:20 PM, Stefan Koch wrote:

 Yes making the function a function template will do that.
 And Manu is fully aware of it.

 It's a nasty hack.
 And in certain cases it screws with overload-resolution.


 It's not exactly the same. If the compiler thinks an imported module 
 has instantiated it, it will not include it in the object file (unless 
 you use -allinst).


 
 Nope, -allinst doesn't make sure all templates are instantiated in each 
 object file. I'll link again to 
 https://github.com/ldc-developers/ldc/pull/3422#issuecomment-625945508.


I can't tell from that discussion what you mean.

My understanding with -allinst is that the compiler is supposed to 
instantiate all templates even if it thinks a dependent library has 
already done it.

But I'm happy to defer to you as an actual developer of the compiler, 
I'm not. I have had experience as a user with the compiler making wrong 
decisions about whether it should emit code for a template instance or not.

In any case, my point is that just making it a template isn't the 
equivalent as has been suggested.

-Steve
Jun 10 2020
parent reply kinke <noone nowhere.com> writes:
On Wednesday, 10 June 2020 at 19:20:55 UTC, Steven Schveighoffer 
wrote:

 My understanding with -allinst is that the compiler is supposed 
 to instantiate all templates even if it thinks a dependent 
 library has already done it.

 But I'm happy to defer to you as an actual developer of the 
 compiler, I'm not. I have had experience as a user with the 
 compiler making wrong decisions about whether it should emit 
 code for a template instance or not.

 In any case, my point is that just making it a template isn't 
 the equivalent as has been suggested.


You're spot-on, except for the fact that a template instance is 
emitted into at most 1 object file per compiler command-line. So 
if you compile a static lib with 100 modules in one go, and each 
module instantiates some common template, it's emitted into 
exactly one object file when using -allinst (and possibly 
-unittest too, I don't remember exactly), and *might* be emitted 
into a (single) object file without -allinst (because as you 
said, if the compiler sees another non-compiled imported module 
instantiating it, the compiler elides its codegen and depends on 
that external definition later when linking).

TLDR: Templates, with the current emission scheme, can definitely 
not be used to emit a function into every referencing CU.
Jun 10 2020
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/10/2020 1:10 PM, kinke wrote:

 TLDR: Templates, with the current emission scheme, can definitely not be used
to 
 emit a function into every referencing CU.


Yes, but in Manu's case the idea is to not need to link with a library. It's 
hard to see how the compiler would know that a template was instantiated in a 
library - meaning it'll emit an instance for at least one of the modules given 
on the command line.

That should resolve the particular issue Manu was having.
Jun 11 2020
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 11 June 2020 at 10:05:14 UTC, Walter Bright wrote:

 On 6/10/2020 1:10 PM, kinke wrote:

 TLDR: Templates, with the current emission scheme, can 
 definitely not be used to emit a function into every 
 referencing CU.


 Yes, but in Manu's case the idea is to not need to link with a 
 library. It's hard to see how the compiler would know that a 
 template was instantiated in a library - meaning it'll emit an 
 instance for at least one of the modules given on the command 
 line.

 That should resolve the particular issue Manu was having.


The Idea is to be able to have a "header-only" library where 
importing is enough to use it.
Which will only (transitively) draw in  the functionality used.
While it is possible with nullary templates it is dissatisfying, 
dissatisfying enough to cause a busy person like Manu to write a 
post about it.
Jun 11 2020
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 6/11/20 6:05 AM, Walter Bright wrote:

 On 6/10/2020 1:10 PM, kinke wrote:

 TLDR: Templates, with the current emission scheme, can definitely not 
 be used to emit a function into every referencing CU.


 
 Yes, but in Manu's case the idea is to not need to link with a library. 
 It's hard to see how the compiler would know that a template was 
 instantiated in a library - meaning it'll emit an instance for at least 
 one of the modules given on the command line.
 
 That should resolve the particular issue Manu was having.


An example:

--- b.d:
module b;

int foo(T)()
{
    return 1;
}

enum x = foo!int();

--- a.d:
import b;

void main()
{
    auto  = foo!int();
}
---


 dmd a.d


Undefined symbols for architecture x86_64:
   "__D1b__T3fooTiZQhFNaNbNiNfZi", referenced from:
       __Dmain in a.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see 
invocation)
Error: linker exited with status 1

comment out the enum and it works.

-Steve
Jun 11 2020
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/11/2020 8:16 AM, Steven Schveighoffer wrote:

 comment out the enum and it works.


Sounds good to me. Don't do that in the "header" file and it'll work.
Jun 11 2020
prev sibling parent reply kinke <noone nowhere.com> writes:
On Thursday, 11 June 2020 at 10:05:14 UTC, Walter Bright wrote:

 On 6/10/2020 1:10 PM, kinke wrote:

 TLDR: Templates, with the current emission scheme, can 
 definitely not be used to emit a function into every 
 referencing CU.


 Yes, but in Manu's case the idea is to not need to link with a 
 library.


I know, just saying that the workaround suggested by Stefan 
('just make it a dummy template') doesn't work in general - it 
*can* work, but only when using -allinst in combination with 
compiling a single object file per compiler invocation, basically 
excluding all static libraries compiled in one cmdline (dub 
default, right?).

In the linked LDC PR, I'm experimenting with a different template 
emission scheme, emitting each referenced template stuff into 
each referencing CU/object file, using linkonce_odr linkage. As a 
side effect, that would enable the template workaround (or open 
the door for some other syntactic way of emitting regular 
non-templated functions the same way), without -allinst or having 
to take care of how the compiler is invoked.

My interest in this wasn't triggered by wanting to enforce 
particular functions to be emitted into each referencing CU, but 
by
a) the current emission scheme requiring LTO in order to inline 
suited little templated functions when compiling static libs in 
one go,
b) being able to use linkonce_odr instead of weak_odr linkage for 
template stuff, meaning potentially less work for the linker, and 
potentially quite a bit less work for the optimizer. Compiling an 
optimized dub (a single object file) with LDC using 
-linkonce-templates reduces the total compile+optimize times by 
roughly 25%, simply because LLVM inlines most little template 
stuff and can then discard the unused linkonce_odr functions 
early in the process, without uselessly optimizing them to death 
(as it currently cannot discard it, as other libraries/objects 
might depend on some template instances as mentioned earlier).
Jun 11 2020
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/11/2020 8:42 AM, kinke wrote:

 My interest in this wasn't triggered by wanting to enforce particular
functions 
 to be emitted into each referencing CU, but by
 a) the current emission scheme requiring LTO in order to inline suited little 
 templated functions when compiling static libs in one go,
 b) being able to use linkonce_odr instead of weak_odr linkage for template 
 stuff, meaning potentially less work for the linker, and potentially quite a
bit 
 less work for the optimizer. Compiling an optimized dub (a single object file) 
 with LDC using -linkonce-templates reduces the total compile+optimize times by 
 roughly 25%, simply because LLVM inlines most little template stuff and can
then 
 discard the unused linkonce_odr functions early in the process, without 
 uselessly optimizing them to death (as it currently cannot discard it, as
other 
 libraries/objects might depend on some template instances as mentioned
earlier).


A library consists of two parts:

1. the "header" file
2. the binary to link with, which may be absent

Take some care in what goes in 1, what goes in 2, and you'll be fine.

For example, consider core.checkedint. The functions in it are all templates so 
that they can be used with -betterC which doesn't link with druntime. It's a 
classic "header only" library. It works fine.
Jun 11 2020
next sibling parent Manu <turkeyman gmail.com> writes:
On Fri, Jun 12, 2020 at 8:20 AM Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 6/11/2020 8:42 AM, kinke wrote:

 My interest in this wasn't triggered by wanting to enforce particular


 functions

 to be emitted into each referencing CU, but by
 a) the current emission scheme requiring LTO in order to inline suited


 little

 templated functions when compiling static libs in one go,
 b) being able to use linkonce_odr instead of weak_odr linkage for


 template

 stuff, meaning potentially less work for the linker, and potentially


 quite a bit

 less work for the optimizer. Compiling an optimized dub (a single object


 file)

 with LDC using -linkonce-templates reduces the total compile+optimize


 times by

 roughly 25%, simply because LLVM inlines most little template stuff and


 can then

 discard the unused linkonce_odr functions early in the process, without
 uselessly optimizing them to death (as it currently cannot discard it,


 as other

 libraries/objects might depend on some template instances as mentioned


 earlier).

 A library consists of two parts:

 1. the "header" file
 2. the binary to link with, which may be absent

 Take some care in what goes in 1, what goes in 2, and you'll be fine.

 For example, consider core.checkedint. The functions in it are all
 templates so
 that they can be used with -betterC which doesn't link with druntime. It's
 a
 classic "header only" library. It works fine.


We're not talking about templates.
Jun 11 2020
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On Fri, Jun 12, 2020 at 11:51 AM Manu <turkeyman gmail.com> wrote:


 On Fri, Jun 12, 2020 at 8:20 AM Walter Bright via Digitalmars-d <
 digitalmars-d puremagic.com> wrote:


 On 6/11/2020 8:42 AM, kinke wrote:

 My interest in this wasn't triggered by wanting to enforce particular


 functions

 to be emitted into each referencing CU, but by
 a) the current emission scheme requiring LTO in order to inline suited


 little

 templated functions when compiling static libs in one go,
 b) being able to use linkonce_odr instead of weak_odr linkage for


 template

 stuff, meaning potentially less work for the linker, and potentially


 quite a bit

 less work for the optimizer. Compiling an optimized dub (a single


 object file)

 with LDC using -linkonce-templates reduces the total compile+optimize


 times by

 roughly 25%, simply because LLVM inlines most little template stuff and


 can then

 discard the unused linkonce_odr functions early in the process, without
 uselessly optimizing them to death (as it currently cannot discard it,


 as other

 libraries/objects might depend on some template instances as mentioned


 earlier).

 A library consists of two parts:

 1. the "header" file
 2. the binary to link with, which may be absent

 Take some care in what goes in 1, what goes in 2, and you'll be fine.

 For example, consider core.checkedint. The functions in it are all
 templates so
 that they can be used with -betterC which doesn't link with druntime.
 It's a
 classic "header only" library. It works fine.


 We're not talking about templates.


This is another one of those cases is really bizarre resistance to what
seems like a blindingly obvious thing.
Can you please explain why you feel opposed to this, and why our existing
solution, which is 'weird' by all accounts, and satisfies ZERO of the goals
assigned to inline is preferable?
The existing implementation is inline is a COMPLETELY useless thing. If you
can't see where I'm coming from in this thread, then I suggest we remove it
completely.
It's existence solves zero problems, and does actual damage to D by
existing, because people try it and find that it's broken, and then they're
confused and/or lose a little confidence in D.
I mean that seriously; if you deny there's a problem and refuse to fix
this, then I seriously encourage you to deprecate and remove pragma(inline)
from the language. It's worse than nothing.
Jun 11 2020
next sibling parent reply Avrina <avrina12309412342 gmail.com> writes:
On Friday, 12 June 2020 at 01:55:26 UTC, Manu wrote:

 Can you please explain why you feel opposed to this, and why 
 our existing
 solution, which is 'weird' by all accounts, and satisfies ZERO 
 of the goals
 assigned to inline is preferable?


You won't get a direct answer to this. You can't have a debate if 
he never presents an argument. Or rather in business, you don't 
have to deal with a problem if you don't acknowledge there is a 
problem, management 101.
Jun 11 2020
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/11/20 11:01 PM, Avrina wrote:

 On Friday, 12 June 2020 at 01:55:26 UTC, Manu wrote:

 Can you please explain why you feel opposed to this, and why our existing
 solution, which is 'weird' by all accounts, and satisfies ZERO of the 
 goals
 assigned to inline is preferable?


 
 You won't get a direct answer to this. You can't have a debate if he 
 never presents an argument. Or rather in business, you don't have to 
 deal with a problem if you don't acknowledge there is a problem, 
 management 101.


It seems to me the problem lies with the way the problem is formulated.
Jun 12 2020
parent reply Avrina <avrina12309412342 gmail.com> writes:
On Friday, 12 June 2020 at 15:37:16 UTC, Andrei Alexandrescu 
wrote:

 On 6/11/20 11:01 PM, Avrina wrote:

 On Friday, 12 June 2020 at 01:55:26 UTC, Manu wrote:

 Can you please explain why you feel opposed to this, and why 
 our existing
 solution, which is 'weird' by all accounts, and satisfies 
 ZERO of the goals
 assigned to inline is preferable?


 
 You won't get a direct answer to this. You can't have a debate 
 if he never presents an argument. Or rather in business, you 
 don't have to deal with a problem if you don't acknowledge 
 there is a problem, management 101.


 It seems to me the problem lies with the way the problem is 
 formulated.


You managed to figure it out :), at least better than Walter. 
When talking about inline, it seems even you and Walter seem to 
be focused on the "inline" part.

Yes an empty template does what Manu wants, the problem is that 
templates have their own nasty side effects.

    int  foo(int)  { return 1; }
    bool foo(bool) { return true; }

    int  bar()(int)  { return 1; }
    bool bar()(bool) { return true; }

    pragma(msg, __traits(getOverloads, __traits(parent, foo), 
"foo")); // tuple of (foo, foo)
    pragma(msg, __traits(getOverloads, __traits(parent, bar), 
"bar")); // empty tuple ()


It's relevant enough to be a feature that has already existed for 
quite some time. All Manu wants is to have that same 
functionality without the template. Maybe a bit different of an 
implementation but it does almost the same thing. There were also 
a few other examples of problems provided that were meant with 
unhelpful "workarounds" such as this: 
https://forum.dlang.org/post/rbua3s$1s5m$1 digitalmars.com Yes, 
just don't ever use enums and/or waste your time trying to figure 
out what is causing the link error and try to figure out a way to 
just remove it. So helpful.
Jun 12 2020
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/12/20 4:13 PM, Avrina wrote:

 On Friday, 12 June 2020 at 15:37:16 UTC, Andrei Alexandrescu wrote:

 On 6/11/20 11:01 PM, Avrina wrote:

 On Friday, 12 June 2020 at 01:55:26 UTC, Manu wrote:

 Can you please explain why you feel opposed to this, and why our 
 existing
 solution, which is 'weird' by all accounts, and satisfies ZERO of 
 the goals
 assigned to inline is preferable?


 You won't get a direct answer to this. You can't have a debate if he 
 never presents an argument. Or rather in business, you don't have to 
 deal with a problem if you don't acknowledge there is a problem, 
 management 101.


 It seems to me the problem lies with the way the problem is formulated.


 
 You managed to figure it out :), at least better than Walter. When 
 talking about inline, it seems even you and Walter seem to be focused on 
 the "inline" part.
 
 Yes an empty template does what Manu wants, the problem is that 
 templates have their own nasty side effects.
 
     int  foo(int)  { return 1; }
     bool foo(bool) { return true; }
 
     int  bar()(int)  { return 1; }
     bool bar()(bool) { return true; }
 
     pragma(msg, __traits(getOverloads, __traits(parent, foo), "foo")); 
 // tuple of (foo, foo)
     pragma(msg, __traits(getOverloads, __traits(parent, bar), "bar")); 
 // empty tuple ()
 
 
 It's relevant enough to be a feature that has already existed for quite 
 some time. All Manu wants is to have that same functionality without the 
 template.


Thanks, that's edifying. That makes sense - have pragma(inline, true) do 
what a template would, without actually using template instantiation.


 Maybe a bit different of an implementation but it does almost 
 the same thing.


This is important. Again it's been a repeating pattern that a valid 
request comes with a barrage of red herring details and suggestions for 
implementation that are often incomplete, unnecessarily complex, 
difficult to implement, or just plain wrong.


 There were also a few other examples of problems 
 provided that were meant with unhelpful "workarounds" such as this: 
 https://forum.dlang.org/post/rbua3s$1s5m$1 digitalmars.com Yes, just 
 don't ever use enums and/or waste your time trying to figure out what is 
 causing the link error and try to figure out a way to just remove it. So 
 helpful.


I confess I don't understand why the example doesn't work. It should. 
And the "doctor, it hurts when I do that - then don't do that" 
back-and-forth is not helping.
Jun 12 2020
prev sibling parent Manu <turkeyman gmail.com> writes:
On Sat, Jun 13, 2020 at 6:15 AM Avrina via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Friday, 12 June 2020 at 15:37:16 UTC, Andrei Alexandrescu
 wrote:

 On 6/11/20 11:01 PM, Avrina wrote:

 On Friday, 12 June 2020 at 01:55:26 UTC, Manu wrote:

 Can you please explain why you feel opposed to this, and why
 our existing
 solution, which is 'weird' by all accounts, and satisfies
 ZERO of the goals
 assigned to inline is preferable?


 You won't get a direct answer to this. You can't have a debate
 if he never presents an argument. Or rather in business, you
 don't have to deal with a problem if you don't acknowledge
 there is a problem, management 101.


 It seems to me the problem lies with the way the problem is
 formulated.


 You managed to figure it out :), at least better than Walter.
 When talking about inline, it seems even you and Walter seem to
 be focused on the "inline" part.

 Yes an empty template does what Manu wants, the problem is that
 templates have their own nasty side effects.

     int  foo(int)  { return 1; }
     bool foo(bool) { return true; }

     int  bar()(int)  { return 1; }
     bool bar()(bool) { return true; }

     pragma(msg, __traits(getOverloads, __traits(parent, foo),
 "foo")); // tuple of (foo, foo)
     pragma(msg, __traits(getOverloads, __traits(parent, bar),
 "bar")); // empty tuple ()


Right, this is but one of an endless sea of edge cases extending from the
fact that templates are not functions, they are templates... they may
become functions when they are instantiated, but that requires
instantiation.
Basically the only way that a nullary template function is like a normal
function is a direct call: `foo()`, almost anything else you do has
sufficient semantic differences that break interactions with generic code.

Eg:
    void foo() {}
    void bar()() {}

    // this works (this is the only thing that works):
    foo();
    bar();

    // this doesn't:
    auto x = &foo;
    auto y = &bar;

    // this doesn't:
    static assert(is(typeof(foo) == function));
    static assert(is(typeof(bar) == function));

    // this doesn't:
    pragma(msg, __traits(getOverloads, __traits(parent, foo), "foo")); //
(foo)
    pragma(msg, __traits(getOverloads, __traits(parent, bar), "bar")); // ()

    // etc...

Dummy templates are not a cool workaround. Sometimes functions are just
meant to be functions.
What the template hack does show us though, is that DMD is already
perfectly capable of implementing `inline`, it's probably a 1-line fix.
Jun 12 2020
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/11/20 9:55 PM, Manu wrote:

 This is another one of those cases is really bizarre resistance to what 
 seems like a blindingly obvious thing.


After perusing this entire thread, I must confess I'm unclear of what 
the request is. I'm sure it's something reasonable, and that the 
misunderstanding is mine.

Can someone summarize what the matter is and what the motivation behind 
it is?

Also, starting from the pragma definition: 
https://dlang.org/spec/pragma.html#inline. It's very thin! There's 
definitely a need to improve it. It would need at least enough detail to 
describe what the linkage of inline function is.
Jun 12 2020
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/11/2020 6:55 PM, Manu wrote:

     We're not talking about templates. 
 
 This is another one of those cases is really bizarre resistance to what seems 
 like a blindingly obvious thing.
 Can you please explain why you feel opposed to this, and why our existing 
 solution, which is 'weird' by all accounts, and satisfies ZERO of the goals 
 assigned to inline is preferable?


Specifying solutions without explaining what the problem is means I will never 
understand what you want. For example, I had to keep saying why? why? why? to 
you over several posts before you finally said what you wanted was a header
only 
library.

I show you how you can do that with existing D, along with an existing, working 
solution.

Then you say you're not talking about templates.

Hence I still don't know what problem(s) you are having, as I have no idea why 
templates are no good for you.


 The existing implementation is inline is a COMPLETELY useless thing.


I use it, and it's not useless.

I suggest coming up with a complete list of what problems you are trying to 
solve, not a list of proposed solutions.

BTW, the pragma(inline) came out of a conversation you & I had at a DConf some 
years ago, and we talked about inline and not inlining. I don't recall you ever 
mentioning issues with linkage, multiple instantiations, etc.

I also have problems understanding what you want when things are distributed 
piecemeal over a large number of n.g. posts. I suggest again:

1. putting each problem into a bugzilla issue
2. keeping a text file of URLs to those issues
3. cut & paste that list as necessary and where necessary

N.g. postings are utterly disorganized and ephemeral. Using targeted bugzilla 
issues is a far better way of organizing specific topics for change.
Jun 12 2020
next sibling parent reply Manu <turkeyman gmail.com> writes:
On Sat, Jun 13, 2020 at 11:31 AM Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 6/11/2020 6:55 PM, Manu wrote:

     We're not talking about templates.

 This is another one of those cases is really bizarre resistance to what


 seems

 like a blindingly obvious thing.
 Can you please explain why you feel opposed to this, and why our


 existing

 solution, which is 'weird' by all accounts, and satisfies ZERO of the


 goals

 assigned to inline is preferable?


 Specifying solutions without explaining what the problem is means I will
 never
 understand what you want. For example, I had to keep saying why? why? why?
 to
 you over several posts before you finally said what you wanted was a
 header only
 library.


I felt like my OP was really clear. I identify 3 use cases for inline, and
then commented on their level of importance.
I've answered your "why"s so many times.


'inline')

judgement (C++ calls this `__forceinline` among various others, completely
non-standard, but useful)

can't (C++ doesn't have this at all. It's useful for preserving features of
the calling context)

I show you how you can do that with existing D, along with an existing,

 working
 solution.


Templates are not a solution, they're a weak workaround which I'm
completely aware of. This thread exists because we don't want that
workaround.
Templates are not functions, and inserting a template into any generic code
where a function is expected leads to a sea of edge cases.

Then you say you're not talking about templates.


Because we're not talking about templates. Functions are _functions_, they
have a lot of semantic differences from templates.

Hence I still don't know what problem(s) you are having, as I have no idea

 why
 templates are no good for you.



We don't satisfy any of the 3 key goals in my OP.
After reading the various responses in this thread, I could rewrite that
post to be clearer, but I really didn't feel it was that unclear in the
first place.



 The existing implementation is inline is a COMPLETELY useless thing.


 I use it, and it's not useless.


I'm not aware that I've ever wanted inline for any reason other than those
3 points I described. I invited others to add their own use cases if I was
missing some.
You're welcome to add a use case showing how you'd like to use inline...
but it doesn't implement any of my use cases today.

I suggest coming up with a complete list of what problems you are trying to

 solve, not a list of proposed solutions.


I have, that's the OP.

BTW, the pragma(inline) came out of a conversation you & I had at a DConf

 some
 years ago, and we talked about inline and not inlining. I don't recall you
 ever
 mentioning issues with linkage, multiple instantiations, etc.



Notably, that is a case that C/C++ has absolutely no way to express and it
was very interesting to me at the time to have a way to express it.

I think the mistake I made at the time was that I held an underlying
assumption about what inline does do, and presumed that to be a given.
We never discussed what inline does (emit to the calling CU), and I
certainly never said that was incorrect functionality and that we should
change that behaviour.
We didn't discuss it, I presumed it was a given, fundamental to the concept
of inline, and never gave it a second thought. It never occurred to me that
mechanic could come into question.

When I did realise that mechanic was broken though, I mentioned it at a
later time, and it's never been fixed.



any incarnation of inline must have to be useful.

I also have problems understanding what you want when things are

 distributed
 piecemeal over a large number of n.g. posts. I suggest again:

 1. putting each problem into a bugzilla issue
 2. keeping a text file of URLs to those issues
 3. cut & paste that list as necessary and where necessary

 N.g. postings are utterly disorganized and ephemeral. Using targeted
 bugzilla
 issues is a far better way of organizing specific topics for change.


I agree, my bugzilla issues on inline are confused. I'll straighten them
out.



 Jun 12 2020



  parent  welkam <wwwelkam gmail.com>  writes:


On Saturday, 13 June 2020 at 01:58:55 UTC, Manu wrote:

 On Sat, Jun 13, 2020 at 11:31 AM Walter Bright via 
 Digitalmars-d < digitalmars-d puremagic.com> wrote:

 For example, I had to keep saying why? why? why?


 I've answered your "why"s so many times.
 <...>

 judgement


Oh cmon, Manu. This does not answer why. This is what statement 
that has "why?" before it.
WHAT:
Users want to add a hint to override the optimisers judgement
WHY:
Inlining has a big effect on programs performance and compiler`s 
heuristics based inlining algorithm doesn't always make the right 
decision. Manually going through the code base and marking select 
functions with a tag that tells inliner whether to inline that 
function or not can improve performance by several percent. In 
areas where high performance is critical and you have gain a lot 
by extracting even 1% of performance a tag that controls inliner 
is absolutely necessary. There fore D needs a way to control 
which function gets inlined and which is not otherwise D wont be 
even considered for writing code in high performance computing 
applications.

Now do that for other 2.
P.s. Remember (X) and (Y)?



 Jun 13 2020



prev sibling  parent  Manu <turkeyman gmail.com>  writes:


On Sat, Jun 13, 2020 at 11:58 AM Manu <turkeyman gmail.com> wrote:


 On Sat, Jun 13, 2020 at 11:31 AM Walter Bright via Digitalmars-d <
 digitalmars-d puremagic.com> wrote:


 I also have problems understanding what you want when things are
 distributed


 piecemeal over a large number of n.g. posts. I suggest again:

 1. putting each problem into a bugzilla issue
 2. keeping a text file of URLs to those issues
 3. cut & paste that list as necessary and where necessary

 N.g. postings are utterly disorganized and ephemeral. Using targeted
 bugzilla
 issues is a far better way of organizing specific topics for change.


 I agree, my bugzilla issues on inline are confused. I'll straighten them
 out.


Let's get this right first:
https://issues.dlang.org/show_bug.cgi?id=20925


they both depend on that as a baseline requirement.



 Jun 12 2020



prev sibling  parent  Avrina <avrina12309412342 gmail.com>  writes:


On Tuesday, 9 June 2020 at 09:29:47 UTC, Walter Bright wrote:

 On 6/8/2020 7:09 AM, Manu wrote:

 On Mon, Jun 8, 2020 at 8:20 PM Walter Bright via Digitalmars-d 
 It's not a hint at all. It's a mechanical tool; it marks 
 symbols with internal linkage, and it also doesn't emit them 
 if it's never referenced.
 The compiler may not choose to ignore that behaviour,


 The C/C++ inline semantics revolve around the mechanics of .h 
 files because it doesn't have modules. These reasons are 
 irrelevant for D.


 it's absolutely necessary, and very important.


 For .h files, sure. Why for D, though?


The same is true for .di files. There's no option for the 
compiler to tell the compiler to include the source of a function 
for the header files in D. So there's no chance for it to inline. 
I know .di files aren't used much, but they are still supported 
nonetheless.

Note: This is different than the argument Manu is making, but it 
is as relevant for D as it is for C++ for the compiler to 
actually be able to inline the source.



 Jun 09 2020



prev sibling  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 6/8/20 10:09 AM, Manu wrote:

 It's not a hint at all. It's a mechanical tool; it marks symbols with 
 internal linkage, and it also doesn't emit them if it's never referenced.
 The compiler may not choose to ignore that behaviour, it's absolutely 
 necessary, and very important.


Manu, your understanding of inlining is very wrong. Maybe you refer to 
C's __inline__ (with which I'm not familiar)?



 Jun 12 2020



  parent reply Patrick Schluter <Patrick.Schluter bbox.fr>  writes:


On Friday, 12 June 2020 at 13:27:23 UTC, Andrei Alexandrescu 
wrote:

 On 6/8/20 10:09 AM, Manu wrote:

 It's not a hint at all. It's a mechanical tool; it marks 
 symbols with internal linkage, and it also doesn't emit them 
 if it's never referenced.
 The compiler may not choose to ignore that behaviour, it's 
 absolutely necessary, and very important.


 Manu, your understanding of inlining is very wrong. Maybe you 
 refer to C's __inline__ (with which I'm not familiar)?


Since C99 it's as you've described above. Before that it was 
implementation defined as it was not part of the language but 
compiler dependent.
The behaviour was very different between gcc 3 and gcc 4, after 
that it was as you described above.
I came up with a complex scheme of macros to be able to handle 
inlining efficiently, and portably for our project that was (and 
still is) pure C.

The issue with inlining for a project like ours, is that we have 
a set of libraries and binaries that a linked against these 
libraries. Some of the apps have to be linked to static versions 
of the libs, other to .so version of them, this is 3 variants, 
optimized for deployment, debug with a lot of diagnistics and 
profiling.

This requires that all inline functions have their code emitted 
exactly once in exactly one library, regardless if the caller 
inlined it or not. Not as easy as people think.



 Jun 12 2020



  parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 6/12/20 9:57 AM, Patrick Schluter wrote:

 On Friday, 12 June 2020 at 13:27:23 UTC, Andrei Alexandrescu wrote:

 On 6/8/20 10:09 AM, Manu wrote:

 It's not a hint at all. It's a mechanical tool; it marks symbols with 
 internal linkage, and it also doesn't emit them if it's never 
 referenced.
 The compiler may not choose to ignore that behaviour, it's absolutely 
 necessary, and very important.


 Manu, your understanding of inlining is very wrong. Maybe you refer to 
 C's __inline__ (with which I'm not familiar)?


 
 Since C99 it's as you've described above. Before that it was 
 implementation defined as it was not part of the language but compiler 
 dependent.
 The behaviour was very different between gcc 3 and gcc 4, after that it 
 was as you described above.
 I came up with a complex scheme of macros to be able to handle inlining 
 efficiently, and portably for our project that was (and still is) pure C.
 
 The issue with inlining for a project like ours, is that we have a set 
 of libraries and binaries that a linked against these libraries. Some of 
 the apps have to be linked to static versions of the libs, other to .so 
 version of them, this is 3 variants, optimized for deployment, debug 
 with a lot of diagnistics and profiling.
 
 This requires that all inline functions have their code emitted exactly 
 once in exactly one library, regardless if the caller inlined it or not. 
 Not as easy as people think.


Yes, such batteries of macros are the norm in large projects. I'm 
familiar with those in the HHVM project: 
https://github.com/facebook/hhvm/blob/a0ca1ffa9a3d690ad57feadf27031886f8eeb2f7/hphp/util/portability.h



 Jun 12 2020



prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx>  writes:


On Tue, Jun 09, 2020 at 12:09:04AM +1000, Manu via Digitalmars-d wrote:
[...]

    I don't want a binary full of code that shouldn't be there. It's
    very important to be able to control what code is in your binaries.
    If it's not referenced, it doesn't exist.


[...]

Could you just use LTO for this?  LDC's LTO, for example, lets the
linker discard unreferenced symbols.


T

-- 
A mathematician is a device for turning coffee into theorems. -- P. Erdos



 Jun 08 2020



  parent reply Stanislav Blinov <stanislav.blinov gmail.com>  writes:


On Monday, 8 June 2020 at 14:45:46 UTC, H. S. Teoh wrote:


 Could you just use LTO for this?  LDC's LTO, for example, lets 
 the linker discard unreferenced symbols.


LTO is a tool that attempts to solve a problem that does not need 
to exist. This could be said about linkers in general.



 Jun 08 2020



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 6/8/2020 7:54 AM, Stanislav Blinov wrote:

 On Monday, 8 June 2020 at 14:45:46 UTC, H. S. Teoh wrote:
 

 Could you just use LTO for this?  LDC's LTO, for example, lets the linker 
 discard unreferenced symbols.


 
 LTO is a tool that attempts to solve a problem that does not need to exist.
This 
 could be said about linkers in general.


In the separate compilation model, compilers know nothing about what other 
compilation units may or may not call. This is why elision of unreferenced 
symbols belongs in the linker.



 Jun 09 2020



  parent reply Stanislav Blinov <stanislav.blinov gmail.com>  writes:


On Tuesday, 9 June 2020 at 10:26:45 UTC, Walter Bright wrote:

 On 6/8/2020 7:54 AM, Stanislav Blinov wrote:

 On Monday, 8 June 2020 at 14:45:46 UTC, H. S. Teoh wrote:
 

 Could you just use LTO for this?  LDC's LTO, for example, 
 lets the linker discard unreferenced symbols.


 
 LTO is a tool that attempts to solve a problem that does not 
 need to exist. This could be said about linkers in general.


 In the separate compilation model, compilers know nothing about 
 what other compilation units may or may not call. This is why 
 elision of unreferenced symbols belongs in the linker.


That is no reason for another [instance of the same] compiler not 
to have access to symbol table, and some form of source 
representation, at compile time, obviating the need for (much of) 
linker's work. LTO does what compilers ought to be doing.



 Jun 09 2020



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 6/9/2020 4:30 AM, Stanislav Blinov wrote:

 That is no reason for another [instance of the same] compiler not to have
access 
 to symbol table,


Since the linker has this information already, there's no reason to not do the 
job in the linker.



 Jun 09 2020



 next sibling parent reply Stanislav Blinov <stanislav.blinov gmail.com>  writes:


On Tuesday, 9 June 2020 at 21:26:38 UTC, Walter Bright wrote:

 On 6/9/2020 4:30 AM, Stanislav Blinov wrote:

 That is no reason for another [instance of the same] compiler 
 not to have access to symbol table,


 Since the linker has this information already, there's no 
 reason to not do the job in the linker.


So, "linkers have to do it because linkers have to do it". Let's 
just stick to outdated tech forever.



 Jun 09 2020



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 6/9/2020 2:54 PM, Stanislav Blinov wrote:

 So, "linkers have to do it because linkers have to do it". Let's just stick to 
 outdated tech forever.


I have seriously considered building the linker into DMD, but don't have the 
resources to do it. If you want to do it, feel free to.



 Jun 09 2020



prev sibling  parent reply Manu <turkeyman gmail.com>  writes:


On Wed, Jun 10, 2020 at 7:30 AM Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 6/9/2020 4:30 AM, Stanislav Blinov wrote:

 That is no reason for another [instance of the same] compiler not to


 have access

 to symbol table,


 Since the linker has this information already, there's no reason to not do
 the
 job in the linker.


Redundant work, undesired object bloat, and various potential link
issues/errors.



 Jun 09 2020



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 6/9/2020 4:31 PM, Manu wrote:

 Redundant work,


It isn't redundant work to do it in the linker. It is redundant to do half of 
the linker's job in the compiler, throw away the other half of it, and re-do it 
in the linker.


 undesired object bloat,


Not a problem since we moved on from floppy disks.


 and various potential link issues/errors.


I suppose that fits in with the common notion that the linker is black magic 
full of trolls and dragons. It isn't, it's a boringly simple program. Although 
it is true that probably only 1 in 100 programmers can explain what a linker
does.

(Most of the complexity of linkers is not inherent, it is the result of file 
formats designed by hamsters and workarounds for compiler and loader bugs. You 
have to pity the linker developer - there's no glamor, nobody understands what 
they do, nobody praises them, they just dump on them with "what does 'undefined 
symbol mean' - must be a bug in the linker" questions.)



 Jun 09 2020



 next sibling parent reply Avrina <avrina12309412342 gmail.com>  writes:


On Wednesday, 10 June 2020 at 01:48:20 UTC, Walter Bright wrote:

 On 6/9/2020 4:31 PM, Manu wrote:

 Redundant work,


 It isn't redundant work to do it in the linker. It is redundant 
 to do half of the linker's job in the compiler, throw away the 
 other half of it, and re-do it in the linker.


 undesired object bloat,


 Not a problem since we moved on from floppy disks.


People complain about Visual Studio's download size, most of the 
download is library files.

I imagine it is also a problem for high performance computing as 
well. So, yes still a problem.


 and various potential link issues/errors.


 I suppose that fits in with the common notion that the linker 
 is black magic full of trolls and dragons. It isn't, it's a 
 boringly simple program. Although it is true that probably only 
 1 in 100 programmers can explain what a linker does.

 (Most of the complexity of linkers is not inherent, it is the 
 result of file formats designed by hamsters and workarounds for 
 compiler and loader bugs. You have to pity the linker developer 
 - there's no glamor, nobody understands what they do, nobody 
 praises them, they just dump on them with "what does 'undefined 
 symbol mean' - must be a bug in the linker" questions.)


I think maybe new programmers think that, ones that come from 
languages like python or otherwise where they don't have to deal 
with the abomination known as a linker. That's the  thing, the 
way the workflow is setup the problem usually isn't with the 
linker which makes it really annoying to try and find where the 
problem actually is. Especially if it's a compiler bug :).



 Jun 10 2020



 next sibling parent  jmh530 <john.michael.hall gmail.com>  writes:


On Wednesday, 10 June 2020 at 18:03:21 UTC, Avrina wrote:

 [snip]

 I imagine it is also a problem for high performance computing 
 as well. So, yes still a problem.
 [snip]


For HPC, it would only be an issue during development (where you 
can probably do a lot of the work on your own machine), rather 
than during production.



 Jun 10 2020



prev sibling  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 6/10/2020 11:03 AM, Avrina wrote:

 I think maybe new programmers think that,


I've seen near total linker puzzlement from people with long experience with C 
and C++. I don't know how this happens, just that it is commonplace.


 ones that come from languages like 
 python or otherwise where they don't have to deal with the abomination known
as 
 a linker.


"Abomination" is a misunderstanding of what a linker is and does.


 That's the  thing, the way the workflow is setup the problem usually 
 isn't with the linker which makes it really annoying to try and find where the 
 problem actually is.


Understanding the simple process of linking is key to quickly resolving whether 
a problem is linker or compiler.

I wrote a "how linkers work" chapter for Kevlin Henney's book "97 Things Every 
Programmer Should Know":

https://www.amazon.com/Things-Every-Programmer-Should-Know/dp/0596809484/

No, I don't get any payments from that book. I was talking to Kevlin once 
grousing about nobody understanding linkers, and he invited me to write a 
chapter :-)



 Jun 10 2020



  parent reply matheus <matheus gmail.com>  writes:


On Wednesday, 10 June 2020 at 22:29:59 UTC, Walter Bright wrote:

 ...
 I wrote a "how linkers work" chapter for Kevlin Henney's book 
 "97 Things Every Programmer Should Know":

 https://www.amazon.com/Things-Every-Programmer-Should-Know/dp/0596809484/

 No, I don't get any payments from that book. I was talking to 
 Kevlin once grousing about nobody understanding linkers, and he 
 invited me to write a chapter :-)


Interesting: 
https://github.com/97-things/97-things-every-programmer-should-know/blob/master/en/thing_53/README.md

Matheus.



 Jun 10 2020



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 6/10/2020 3:47 PM, matheus wrote:

 Interesting: 
 https://github.com/97-things/97-things-every-programmer-should-know/blob/master/e
/thing_53/README.md 


Haha, I didn't know it was free online! Thanks for the link, and enjoy!



 Jun 10 2020



prev sibling next sibling parent  Stefan Koch <uplink.coder googlemail.com>  writes:


On Wednesday, 10 June 2020 at 01:48:20 UTC, Walter Bright wrote:

 On 6/9/2020 4:31 PM, Manu wrote:

 [...]


 It isn't redundant work to do it in the linker. It is redundant 
 to do half of the linker's job in the compiler, throw away the 
 other half of it, and re-do it in the linker.

 [...]


we have a project that compiles with "-allinst -allinst -allinst".
To avoid such errors.
(the repetition comes from dub appending to the dflags)



 Jun 10 2020



prev sibling  parent reply Stanislav Blinov <stanislav.blinov gmail.com>  writes:


On Wednesday, 10 June 2020 at 01:48:20 UTC, Walter Bright wrote:

 On 6/9/2020 4:31 PM, Manu wrote:

 Redundant work,


 It isn't redundant work to do it in the linker. It is redundant 
 to do half of the linker's job in the compiler, throw away the 
 other half of it, and re-do it in the linker.


It is redundant to do the job in the compiler, throw *that* away 
and then shrug it off with "meh, the linker will take care of 
it". Which is precisely what compilers are doing. Which is 
precisely what you suggested to be doing in the issue report.
...And of course, the linker doesn't take care of it, which is 
why LTO came to be - to solve a problem that doesn't need to 
exist.

And DMD does not need a built-in linker.



 Jun 10 2020



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 6/10/2020 12:04 PM, Stanislav Blinov wrote:

 It is redundant to do the job in the compiler, throw *that* away and then
shrug 
 it off with "meh, the linker will take care of it". Which is precisely what 
 compilers are doing.


All I can say is you're misunderstanding what linkers do.



 Jun 10 2020



prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com>  writes:


On Monday, June 8, 2020 8:09:04 AM MDT Manu via Digitalmars-d wrote:

 On Mon, Jun 8, 2020 at 8:20 PM Walter Bright via Digitalmars-d <

 C/C++ inline has always been a hint to the compiler, not a command.


 It's not a hint at all. It's a mechanical tool; it marks symbols with
 internal linkage, and it also doesn't emit them if it's never referenced.
 The compiler may not choose to ignore that behaviour, it's absolutely
 necessary, and very important.


It is my understanding that in C++, inline is a hint to the compiler with
regards to whether a particular function call is actually inlined. There are
other effects that inline has which aren't just a hint, but the behavior of
actually inlining the function is a hint, not a command.

Of course, in D, it's even weirder, because it's been "give me an error if
the compiler fails to inline it," and I don't know that it even functions as
a hint at that point as opposed to just a way to find out that the compiler
didn't do what you wanted.

- Jonathan M Davis



 Jun 08 2020



  parent reply Atila Neves <atila.neves gmail.com>  writes:


On Monday, 8 June 2020 at 23:19:55 UTC, Jonathan M Davis wrote:

 On Monday, June 8, 2020 8:09:04 AM MDT Manu via Digitalmars-d 
 wrote:

 On Mon, Jun 8, 2020 at 8:20 PM Walter Bright via Digitalmars-d 
 <

 C/C++ inline has always been a hint to the compiler, not a 
 command.


 It's not a hint at all. It's a mechanical tool; it marks 
 symbols with internal linkage, and it also doesn't emit them 
 if it's never referenced. The compiler may not choose to 
 ignore that behaviour, it's absolutely necessary, and very 
 important.


 It is my understanding that in C++, inline is a hint to the 
 compiler with regards to whether a particular function call is 
 actually inlined.


That's a common misconception, and one that exists due to that 
being its original intended purpose. But nowawadays? Nope:

https://en.cppreference.com/w/cpp/language/inline



 Jun 09 2020



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 6/9/2020 4:31 AM, Atila Neves wrote:

 On Monday, 8 June 2020 at 23:19:55 UTC, Jonathan M Davis wrote:

 On Monday, June 8, 2020 8:09:04 AM MDT Manu via Digitalmars-d wrote:

 On Mon, Jun 8, 2020 at 8:20 PM Walter Bright via Digitalmars-d <

 C/C++ inline has always been a hint to the compiler, not a > command.


 It's not a hint at all. It's a mechanical tool; it marks symbols with 
 internal linkage, and it also doesn't emit them if it's never referenced. The 
 compiler may not choose to ignore that behaviour, it's absolutely necessary, 
 and very important.


 It is my understanding that in C++, inline is a hint to the compiler with 
 regards to whether a particular function call is actually inlined.


 
 That's a common misconception, and one that exists due to that being its 
 original intended purpose. But nowawadays? Nope:
 
 https://en.cppreference.com/w/cpp/language/inline



"Since this meaning of the keyword inline is non-binding, compilers are free to 
use inline substitution for any function that's not marked inline, and are free 
to generate function calls to any function marked inline. Those optimization 
choices do not change the rules regarding multiple definitions and shared 
statics listed above."

I.e. it is a hint.



 Jun 09 2020



 next sibling parent reply Manu <turkeyman gmail.com>  writes:


On Wed, Jun 10, 2020 at 7:15 AM Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 6/9/2020 4:31 AM, Atila Neves wrote:

 On Monday, 8 June 2020 at 23:19:55 UTC, Jonathan M Davis wrote:

 On Monday, June 8, 2020 8:09:04 AM MDT Manu via Digitalmars-d wrote:

 On Mon, Jun 8, 2020 at 8:20 PM Walter Bright via Digitalmars-d <

 C/C++ inline has always been a hint to the compiler, not a > command.


 It's not a hint at all. It's a mechanical tool; it marks symbols with
 internal linkage, and it also doesn't emit them if it's never






 referenced. The

 compiler may not choose to ignore that behaviour, it's absolutely






 necessary,

 and very important.


 It is my understanding that in C++, inline is a hint to the compiler




 with

 regards to whether a particular function call is actually inlined.


 That's a common misconception, and one that exists due to that being its
 original intended purpose. But nowawadays? Nope:

 https://en.cppreference.com/w/cpp/language/inline



 "Since this meaning of the keyword inline is non-binding, compilers are
 free to
 use inline substitution for any function that's not marked inline, and are
 free
 to generate function calls to any function marked inline. Those
 optimization
 choices do not change the rules regarding multiple definitions and shared
 statics listed above."

 I.e. it is a hint.


It's like you skipped over ALL OF THE OTHER TEXT, where it details numerous
precise behavioural requirements :/



 Jun 09 2020



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 6/9/2020 4:30 PM, Manu wrote:

 It's like you skipped over ALL OF THE OTHER TEXT, where it details numerous 
 precise behavioural requirements :/


As I replied elsewhere to you, those other requirements are there to deal with 
.h files, obsolete linkers, and ancient object file formats.


 "Because the meaning of the keyword inline for functions came to mean 


"multiple definitions are permitted" rather than "inlining is preferred", that 
meaning was extended to variables."

It's a hack for the the same problem - .h files, etc. There's no reason for
that 
in D.



 Jun 09 2020



prev sibling  parent  Manu <turkeyman gmail.com>  writes:


On Wed, Jun 10, 2020 at 9:30 AM Manu <turkeyman gmail.com> wrote:


 On Wed, Jun 10, 2020 at 7:15 AM Walter Bright via Digitalmars-d <
 digitalmars-d puremagic.com> wrote:


 On 6/9/2020 4:31 AM, Atila Neves wrote:

 On Monday, 8 June 2020 at 23:19:55 UTC, Jonathan M Davis wrote:

 On Monday, June 8, 2020 8:09:04 AM MDT Manu via Digitalmars-d wrote:

 On Mon, Jun 8, 2020 at 8:20 PM Walter Bright via Digitalmars-d <

 C/C++ inline has always been a hint to the compiler, not a >








 command.

 It's not a hint at all. It's a mechanical tool; it marks symbols with
 internal linkage, and it also doesn't emit them if it's never






 referenced. The

 compiler may not choose to ignore that behaviour, it's absolutely






 necessary,

 and very important.


 It is my understanding that in C++, inline is a hint to the compiler




 with

 regards to whether a particular function call is actually inlined.


 That's a common misconception, and one that exists due to that being


 its

 original intended purpose. But nowawadays? Nope:

 https://en.cppreference.com/w/cpp/language/inline



 "Since this meaning of the keyword inline is non-binding, compilers are
 free to
 use inline substitution for any function that's not marked inline, and
 are free
 to generate function calls to any function marked inline. Those
 optimization
 choices do not change the rules regarding multiple definitions and shared
 statics listed above."

 I.e. it is a hint.


 It's like you skipped over ALL OF THE OTHER TEXT, where it details
 numerous precise behavioural requirements :/


The the sentence that immediately follows your quote is:
"Because the meaning of the keyword inline for functions came to mean
"multiple definitions are permitted" rather than "inlining is preferred",
that meaning was extended to variables."



 Jun 09 2020



prev sibling next sibling parent  Manu <turkeyman gmail.com>  writes:


On Tue, Jun 9, 2020 at 12:46 AM H. S. Teoh via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Tue, Jun 09, 2020 at 12:09:04AM +1000, Manu via Digitalmars-d wrote:
 [...]

    I don't want a binary full of code that shouldn't be there. It's
    very important to be able to control what code is in your binaries.
    If it's not referenced, it doesn't exist.


 [...]

 Could you just use LTO for this?  LDC's LTO, for example, lets the
 linker discard unreferenced symbols.


I think the compiler should specify behaviour that's universally
compatible, and doesn't depend on special tooling to integrate with
existing workflows.



 Jun 08 2020



prev sibling next sibling parent  Manu <turkeyman gmail.com>  writes:


On Tue, Jun 9, 2020 at 9:20 AM Jonathan M Davis via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Monday, June 8, 2020 8:09:04 AM MDT Manu via Digitalmars-d wrote:

 On Mon, Jun 8, 2020 at 8:20 PM Walter Bright via Digitalmars-d <

 C/C++ inline has always been a hint to the compiler, not a command.


 It's not a hint at all. It's a mechanical tool; it marks symbols with
 internal linkage, and it also doesn't emit them if it's never referenced.
 The compiler may not choose to ignore that behaviour, it's absolutely
 necessary, and very important.


 It is my understanding that in C++, inline is a hint to the compiler with
 regards to whether a particular function call is actually inlined. There
 are
 other effects that inline has which aren't just a hint, but the behavior of
 actually inlining the function is a hint, not a command.


I'm not sure it's even a hint. That behaviour is basically irrelevant. The
'other effects' you describe are what inline does in C++, and we need that
too.



 Jun 08 2020



prev sibling  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 6/8/2020 3:19 AM, Walter Bright wrote:

 The PR I have on this makes it an informational warning. You can choose to be 
 notified if inlining fails.


https://github.com/dlang/dmd/pull/11236



 Jun 10 2020



prev sibling next sibling parent reply kinke <kinke gmx.net>  writes:


On Monday, 8 June 2020 at 06:14:44 UTC, Manu wrote:

 D's inline today doesn't do any of those things. It doesn't 
 implement a mechanic that I have ever wanted or known a use for.


We've had this discussion a while back. As of the latest LDC 
beta, `pragma(inline, true)` functions are now properly emitted 
into each referencing CU, and will be inlined in most cases 
(`alwaysinline` LLVM function attribute), even at -O0.

In modules other than the owning one, the function 'copy' is 
emitted as `available_externally`, meaning that it's *only* 
available for inlining at the IR level, it will never make it to 
the assembler and object file.

In its owning module, the function is emitted as a regular 
function, as a fallback for non-inlined cases (and for when 
people take its address etc.). Our opinions diverge wrt. whether 
that's a problem - to me it's clearly no big deal, as the 
function is a) most likely small, and b) subject to linker 
stripping if unreferenced.

Wrt. control of which CUs contain which functions, that's totally 
out of hand anyway due to the way templates are emitted.



 Jun 08 2020



  parent reply Manu <turkeyman gmail.com>  writes:


On Mon, Jun 8, 2020 at 8:30 PM kinke via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Monday, 8 June 2020 at 06:14:44 UTC, Manu wrote:

 D's inline today doesn't do any of those things. It doesn't
 implement a mechanic that I have ever wanted or known a use for.


 We've had this discussion a while back. As of the latest LDC
 beta, `pragma(inline, true)` functions are now properly emitted
 into each referencing CU, and will be inlined in most cases
 (`alwaysinline` LLVM function attribute), even at -O0.


Yes, and thank you; it's always nice that we can fix broken things in LDC.
But I think for inline it's actually quite important that the spec is
useful, and that all compilers do the same thing; otherwise DMD experiences
link errors that LDC properly avoids.

In modules other than the owning one, the function 'copy' is

 emitted as `available_externally`, meaning that it's *only*
 available for inlining at the IR level, it will never make it to
 the assembler and object file.


Yeah I always forget the proper names for all the linkage flags that
symbols can have.
This approach is fine from a linkage point of view (there shouldn't be
errors), but I'd really like to not see the symbol in the owning module if
it's never called locally.
It's a shame to have a ton of noise in a binary when it's unnecessary.
Most D software is full of bloaty junk, but in the cases where I care about
this, it's actually a very tightly controlled binary ecosystem, which binds
between multiple languages, and only exposes necessary stuff at the ABI
boundary.
I've also seen cases where the function is not referenced in the owning
module, but it does not get stripped by the linker for whatever reason, and
then if the inline function has a call to an extern symbol that you don't
link, you'll get link errors, even though there's no calls.
By emitting it even when it's never been referenced, we're just inviting
link errors and inconsistent behaviour between different linkers. We can
trivially avoid that risk.

In its owning module, the function is emitted as a regular

 function, as a fallback for non-inlined cases (and for when
 people take its address etc.). Our opinions diverge wrt. whether
 that's a problem - to me it's clearly no big deal, as the
 function is a) most likely small, and b) subject to linker
 stripping if unreferenced.



a) most likely small, but still not nothing; the symbol table is public ABI
material, and in some projects I've worked on, the symbol table is
carefully curated.
b) linker stripped is not reliable. We are unnecessarily inviting issues in
some cases, and there's just no reason for that.

If we're confident that link stripping is 100% reliable when a symbol is
not referenced, then I have no complaint here.

Can you show what case a hard-symbol in the owning CU solves? Non-inlined
cases will still find it locally if it has internal linkage (or whatever
that link flag is called).
I think it's the same flag that `static` (or `inline`) in C++ specifies
right?

Wrt. control of which CUs contain which functions, that's totally

 out of hand anyway due to the way templates are emitted.


I'm not sure what you mean. Templates work correctly; template instances
are only generated and emit to the calling CU.



 Jun 08 2020



 next sibling parent  kinke <kinke gmx.net>  writes:


On Monday, 8 June 2020 at 14:22:50 UTC, Manu wrote:

 Templates work correctly; template instances are only generated 
 and emit to the calling CU.


Nope - they *might* be emitted into the instantiating module, but 
very likely aren't. Just try compiling a static lib with multiple 
modules instantiating equivalent templates. See 
https://github.com/ldc-developers/ldc/pull/3422#issuecomment-625945508 for an
example.



 Jun 08 2020



prev sibling  parent reply kinke <noone nowhere.com>  writes:


On Monday, 8 June 2020 at 14:22:50 UTC, Manu wrote:

 On Mon, Jun 8, 2020 at 8:30 PM kinke via Digitalmars-d <

 In its owning module, the function is emitted as a regular
 function, as a fallback for non-inlined cases (and for when 
 people take its address etc.). Our opinions diverge wrt. 
 whether that's a problem - to me it's clearly no big deal, as 
 the function is a) most likely small, and b) subject to linker 
 stripping if unreferenced.



 a) most likely small, but still not nothing; the symbol table 
 is public ABI
 material, and in some projects I've worked on, the symbol table 
 is
 carefully curated.
 b) linker stripped is not reliable. We are unnecessarily 
 inviting issues in
 some cases, and there's just no reason for that.

 If we're confident that link stripping is 100% reliable when a 
 symbol is not referenced, then I have no complaint here.

 Can you show what case a hard-symbol in the owning CU solves? 
 Non-inlined
 cases will still find it locally if it has internal linkage (or 
 whatever
 that link flag is called).
 I think it's the same flag that `static` (or `inline`) in C++ 
 specifies
 right?


Internal linkage (C(++) static), AFAIK, means that you may end up 
with multiple identical functions in the final linked binary. 
Some linkers may be able to still fold them, such as the MS 
linker and lld with /OPT:ICF (identical COMDAT folding).

Linkonce_odr linkage (C++ templates and `inline` functions) on 
the other hand means that you end up with either 0 or 1 function 
in the final binary, even when multiple object files define it; 
AFAIK, without having to rely on /OPT:REF or --gc-sections etc. 
At least for LLVM, linkonce_odr additionally means that the 
definition is discarded early if unused and might not make it to 
the object file at all.

Weak_odr linkage (current D templates) is similar to linkonce_odr 
but not 'officially' discardable (if unreferenced) for the final 
binary (and thus always making it to the object file). This can 
be overridden by /OPT:REF and --gc-sections etc., the success of 
which also depends on symbol visibility and/or --export-dynamic 
etc.



 Jun 11 2020



 next sibling parent reply Manu <turkeyman gmail.com>  writes:


On Fri, Jun 12, 2020 at 3:40 AM kinke via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Monday, 8 June 2020 at 14:22:50 UTC, Manu wrote:

 On Mon, Jun 8, 2020 at 8:30 PM kinke via Digitalmars-d <

 In its owning module, the function is emitted as a regular
 function, as a fallback for non-inlined cases (and for when
 people take its address etc.). Our opinions diverge wrt.
 whether that's a problem - to me it's clearly no big deal, as
 the function is a) most likely small, and b) subject to linker
 stripping if unreferenced.



 a) most likely small, but still not nothing; the symbol table
 is public ABI
 material, and in some projects I've worked on, the symbol table
 is
 carefully curated.
 b) linker stripped is not reliable. We are unnecessarily
 inviting issues in
 some cases, and there's just no reason for that.

 If we're confident that link stripping is 100% reliable when a
 symbol is not referenced, then I have no complaint here.

 Can you show what case a hard-symbol in the owning CU solves?
 Non-inlined
 cases will still find it locally if it has internal linkage (or
 whatever
 that link flag is called).
 I think it's the same flag that `static` (or `inline`) in C++
 specifies
 right?


 Internal linkage (C(++) static), AFAIK, means that you may end up
 with multiple identical functions in the final linked binary.
 Some linkers may be able to still fold them, such as the MS
 linker and lld with /OPT:ICF (identical COMDAT folding).


Actually, I might be wrong about what the link flag is called.
I'm fairly sure C++ uses the link flag that LLVM calls "choose one".
It's in the C++ spec that all inlines collapse to the same one. We should
be using the same flag.

Linkonce_odr linkage (C++ templates and `inline` functions) on

 the other hand means that you end up with either 0 or 1 function
 in the final binary, even when multiple object files define it;
 AFAIK, without having to rely on /OPT:REF or --gc-sections etc.
 At least for LLVM, linkonce_odr additionally means that the
 definition is discarded early if unused and might not make it to
 the object file at all.

 Weak_odr linkage (current D templates) is similar to linkonce_odr
 but not 'officially' discardable (if unreferenced) for the final
 binary (and thus always making it to the object file). This can
 be overridden by /OPT:REF and --gc-sections etc., the success of
 which also depends on symbol visibility and/or --export-dynamic
 etc.





 Jun 11 2020



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 6/11/2020 6:51 PM, Manu wrote:

 I'm fairly sure C++ uses the link flag that LLVM calls "choose one".
 It's in the C++ spec that all inlines collapse to the same one. We should be 
 using the same flag.


D totally relies on that behavior.



 Jun 12 2020



  parent reply Manu <turkeyman gmail.com>  writes:


On Sat, Jun 13, 2020 at 11:00 AM Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 6/11/2020 6:51 PM, Manu wrote:

 I'm fairly sure C++ uses the link flag that LLVM calls "choose one".
 It's in the C++ spec that all inlines collapse to the same one. We


 should be

 using the same flag.


 D totally relies on that behavior.


 So emit the function to the CU where it's called and we're done here!



 Jun 12 2020



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 6/12/2020 6:16 PM, Manu wrote:

 On Sat, Jun 13, 2020 at 11:00 AM Walter Bright via Digitalmars-d 
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>> wrote:
 
     On 6/11/2020 6:51 PM, Manu wrote:
      > I'm fairly sure C++ uses the link flag that LLVM calls "choose one".
      > It's in the C++ spec that all inlines collapse to the same one. We
should be
      > using the same flag.
 
     D totally relies on that behavior.
 
 
   So emit the function to the CU where it's called and we're done here!


Next bug report: D takes too long to compile because it's recompiling code it 
has already compiled over and over!

Yes, this has been a bug report in the past. We fixed it.

Besides, I replied to Andrei in another message how you can resolve this - just 
put the 'header' file on the command line to dmd.



 Jun 13 2020



prev sibling  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 6/11/2020 10:39 AM, kinke wrote:

 Internal linkage (C(++) static), AFAIK, means that you may end up with
multiple 
 identical functions in the final linked binary. Some linkers may be able to 
 still fold them, such as the MS linker and lld with /OPT:ICF (identical COMDAT 
 folding).
 
 Linkonce_odr linkage (C++ templates and `inline` functions) on the other hand 
 means that you end up with either 0 or 1 function in the final binary, even
when 
 multiple object files define it; AFAIK, without having to rely on /OPT:REF or 
 --gc-sections etc. At least for LLVM, linkonce_odr additionally means that the 
 definition is discarded early if unused and might not make it to the object
file 
 at all.
 
 Weak_odr linkage (current D templates) is similar to linkonce_odr but not 
 'officially' discardable (if unreferenced) for the final binary (and thus
always 
 making it to the object file). This can be overridden by /OPT:REF and 
 --gc-sections etc., the success of which also depends on symbol visibility 
 and/or --export-dynamic etc.


gc-sections isn't about merging multiple definitions, it's about discarding 
unreferenced sections.



 Jun 12 2020



prev sibling next sibling parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 6/7/2020 11:14 PM, Manu wrote:

 [...]


Please provide URLs to the bugzilla issue(s) you have on this topic.

As I've mentioned before, it's easier if you keep a text file of the bugzilla 
issues that are important to you, so you can copy/paste them where appropriate. 
Otherwise, both of our times are wasted with "did you file a bugzilla issue for 
this?"



 Jun 08 2020



prev sibling next sibling parent  Dukc <ajieskola gmail.com>  writes:


On Monday, 8 June 2020 at 06:14:44 UTC, Manu wrote:

 1. I only want the function to be present in the CALLING 
 binary. I do not want an inline function present in the local 
 binary where it was defined (unless it was called internally). 
 I do not want a linker to see the inline function symbols and 
 be able to link to them externally. [This is about linkage and 
 controlling the binary or distribution environment]


I think this should be controlled by the visibility attributes, 
not `pragma(inline)`. If a function is `private`, there's no 
reason why the linker should see it at all. If it's `public`, it 
should be seen when linking the internal object files together, 
but not when linking a precompiled binary. `export` should always 
be visible.


 2. I am unhappy that the optimiser chose to not inline a 
 function call, and I want to override that judgement. [This is 
 about micro-optimisation]


This is what `pragma(inline)` should be for. For some extent at 
least, it also is. If you compile a simple example with `ldc2 
-O1`, it'll inline the function if you use the pragma but not 
otherwise. `ldc2 -O2` will inline it regardless and DMD will 
never do it IIRC, at least not without the `-inline` switch. I 
agree that it should.


 3. I want to treat the function like an AST macro; I want the 
 function inserted at the callsite, and I want to have total 
 confidence in this mechanic. [This is about articulate 
 mechanical control over code-gen; ie, I know necessary facts 
 about the execution context/callstack that I expect to maintain]


Sounds like a case for mixins, either type of them. But it'd be 
better if the compiler would only ignore `pragma(inline, true)` 
if either linking externally or avoiding recursion. There's just 
no reason to ignore it otherwise. This is an implementation 
problem, not a spec problem IMO.



 Jun 08 2020



prev sibling next sibling parent  Steven Schveighoffer <schveiguy gmail.com>  writes:


On 6/8/20 2:14 AM, Manu wrote:

 3. I want to treat the function like an AST macro; I want the function 
 inserted at the callsite, and I want to have total confidence in this 
 mechanic. [This is about articulate mechanical control over code-gen; 
 ie, I know necessary facts about the execution context/callstack that I 
 expect to maintain]


This is my main use case. Not because I want AST macros (not sure I'd 
agree that AST macro is the right description), but because I write a 
lot of wrapping functions which are like "parameter modifiers". iopipe 
is full of them for instance.

Being able to inject a few statement before/after a call without having 
to dispatch to a separate function first is very useful for a wrapper. 
It's what the optimizer should do anyway, but making it part of the API 
makes a declaration that the function is a wrapper and should be treated 
that way.

Unfortunately, D does not implement this feature in a way that I prefer: 
https://issues.dlang.org/show_bug.cgi?id=15671

-Steve



 Jun 08 2020



prev sibling next sibling parent  9il <ilyayaroshenko gmail.com>  writes:


On Monday, 8 June 2020 at 06:14:44 UTC, Manu wrote:

 1. I only want the function to be present in the CALLING 
 binary. I do not want an inline function present in the local 
 binary where it was defined (unless it was called internally). 
 I do not want a linker to see the inline function symbols and 
 be able to link to them externally. [This is about linkage and 
 controlling the binary or distribution environment]


Yes!


 2. I am unhappy that the optimiser chose to not inline a 
 function call, and I want to override that judgement. [This is 
 about micro-optimisation]


Yes!


 3. I want to treat the function like an AST macro; I want the 
 function inserted at the callsite, and I want to have total 
 confidence in this mechanic. [This is about articulate 
 mechanical control over code-gen; ie, I know necessary facts 
 about the execution context/callstack that I expect to maintain]


Yes!!!


 I think these are the 3 broad categories of behaviour I have 
 ever wanted
 control over.


The same for me.

I have the same experience. Moreover, non-AST inlining has the 
worst optimization abilities comparing with AST. Even if a 
function is inlined it is often inlined badly ignoring some 
optimization attributes, local SIMD and FMA instructions, and 
better loop unrolling patterns (better doesn't mean larger).

AST-like inlining is a critical and killer feature.



 Jun 08 2020



prev sibling  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 6/8/20 2:14 AM, Manu wrote:

 In C/C++, inline says that a function will be emit to the binary only 
 when it is called, and the function is marked with internal linkage (it 
 is not visible to the linker from the symbol table)


By my recollection this is not the case for C++, at all.

* "inline" does NOT change a function's linkage in C++. You may have 
inline functions with internal linkage (static inline) or (default) 
external linkage. This is important because e.g. defining a static 
variable in an extern inline function will have the same address in all 
calls to the function.

* "inline" in C++ is entirely advisory, meaning a conforming 
implementation is entirely free to simply ignore it, save it for 
duplicate symbols (which are ignored for inline functions and cause 
errors for non-inline functions).

* In fact, all major C++ implementations routinely ignore "inline" in 
favor of their own heuristics, which are usually better. However, they 
are not always better, and that in turn has generated demands for an "I 
really mean inline" directive. Compiler implementers responded to that 
demand with a trickle of additional nonstandard language features:

- MSVC has __inline, __forceinline, and __declspec(noinline): 
https://docs.microsoft.com/en-us/cpp/cpp/inline-functions-cpp?view=vs-2019. 
As a funny aside, __inline and __forceinline sometimes do not inline, 
and __declspec(noinline) functions are sometimes inlined.

- g++ and clang have __inline__, __attribute__((always_inline)), and 
__attribute__ ((noinline)): 
https://stackoverflow.com/questions/8381293/how-do-i-force-gcc-t
-inline-a-function. 
Needless to say, __inline__ and __attribute__((always_inline)) do not 
always inline, and __attribute__ ((noinline)) does not always prevent 
inlining.

If the above is correct, a D language feature dedicated to inlining 
should do the following:

* Always emit the function body in the .di file if ever asked to 
generate it.

* Never complain about duplicate symbols if an inline function has 
duplicate definitions.

Then compilers can decide on specific inlining strategies using the 
language feature as a hint.

D does ALL of the above already for templates, so this is not a 
difficult feature to implement. In fact we use this technique (e.g. in 
druntime) by spelling inline as "()". Consider:

int func(int) { ... body ... }

Let's make this inline:

int func()(int) { ... body ... }

Done.

In D there's one additional implication of body availability - the 
function is eligible for CTFE. I think any adjustment to the extant 
inline pragma needs to make this a top-level consideration.



 Jun 12 2020



 next sibling parent reply Johan <j j.nl>  writes:


On Friday, 12 June 2020 at 13:22:35 UTC, Andrei Alexandrescu 
wrote:

 On 6/8/20 2:14 AM, Manu wrote:

 In C/C++, inline says that a function will be emit to the 
 binary only when it is called, and the function is marked with 
 internal linkage (it is not visible to the linker from the 
 symbol table)


 By my recollection this is not the case for C++, at all.

 * "inline" does NOT change a function's linkage in C++. You may 
 have inline functions with internal linkage (static inline) or 
 (default) external linkage. This is important because e.g. 
 defining a static variable in an extern inline function will 
 have the same address in all calls to the function.


I believe Manu tried to explain that `inline` in C++ really only 
affects how the linker must treat the symbol, and it is best to 
remember that it does nothing at all concerning "inlining" 
(putting function body inside another).
`inline` is really a misnomer.
The "linkage" effect of `inline` is required for C++ to work with 
#include files. There is some room for confusion because, 
depending on who is talking, the word "linkage" can have more 
nuances than just internal/external. I personally take the broad 
stance of "linkage" including anything the linker can do with the 
symbol.
Without `inline`, you would get multiple definition linker errors 
for functions defined in a header file when linking together two 
translation units that both included that header file. Instead of 
normal function linkage that forbids multiple definition, 
`inline` changes linkage to merge multiple definitions into one.
In addition, emission of the function must happen in any 
translation unit that references it (calling or address taken), 
and thus that translation unit must also define it (in contrast 
to just declaring it). I do not think `inline` forbids emission 
if the function is not referenced.

Note that C++17 added `inline` for variables too, because without 
it you run into the same problem of multiple definitions in e.g. 
header-only libraries. Obviously that has nothing to do with 
"inlining" the callee's body; it is purely a linker directive to 
merge the symbols which is exactly the functionality it provides 
for functions.

In D, this is actually not a problem: multiple definitions of 
symbols are merged already. You'll never see a multiple 
definition error with D (also not with extern(C) functions).

The wikipedia article seems fairly complete on all the 
intricacies of `inline`: 
https://en.wikipedia.org/wiki/Inline_function

And the cppreference text is also pretty clear  
https://en.cppreference.com/w/cpp/language/inline .

-Johan



 Jun 12 2020



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 6/12/20 2:08 PM, Johan wrote:

 On Friday, 12 June 2020 at 13:22:35 UTC, Andrei Alexandrescu wrote:

 On 6/8/20 2:14 AM, Manu wrote:

 In C/C++, inline says that a function will be emit to the binary only 
 when it is called, and the function is marked with internal linkage 
 (it is not visible to the linker from the symbol table)


 By my recollection this is not the case for C++, at all.

 * "inline" does NOT change a function's linkage in C++. You may have 
 inline functions with internal linkage (static inline) or (default) 
 external linkage. This is important because e.g. defining a static 
 variable in an extern inline function will have the same address in 
 all calls to the function.


 
 I believe Manu tried to explain that `inline` in C++ really only affects 
 how the linker must treat the symbol, and it is best to remember that it 
 does nothing at all concerning "inlining" (putting function body inside 
 another).


Thank you. That's not at all what he wrote. AT ALL. It's what I wrote.

I'm emphasizing this because it has been a recurring problem: a 
legitimate problem with an explanation lost in translation.


 `inline` is really a misnomer.
 The "linkage" effect of `inline` is required for C++ to work with 
 #include files. There is some room for confusion because, depending on 
 who is talking, the word "linkage" can have more nuances than just 
 internal/external.


It helps to use terminology that is well defined and standardized. 
That's where simply using the meaning defined and consistently used in 
the C++ standard document can be of great help.


 I personally take the broad stance of "linkage" 
 including anything the linker can do with the symbol.


Makes sense. It's really nice to just use 
http://eel.is/c++draft/basic.link, which ensures we all understand one 
another.


 Without `inline`, you would get multiple definition linker errors for 
 functions defined in a header file when linking together two translation 
 units that both included that header file. Instead of normal function 
 linkage that forbids multiple definition, `inline` changes linkage to 
 merge multiple definitions into one.


Not sure about that part - if linkage was static by means of using the 
"static" keyword, multiple definitions may not be merged. (I may be 
wrong, please correct me.) Consider:

static inline int fun() {
     static int x;
     return ++x;
}

In C++, each translation unit containing a definition of fun() will have 
a distinct address for x. I don't see how the bodies of those functions 
can be merged.


 In addition, emission of the function must happen in any translation 
 unit that references it (calling or address taken), and thus that 
 translation unit must also define it (in contrast to just declaring it). 
 I do not think `inline` forbids emission if the function is not referenced.


I also think that's the case. (Also not what was claimed.)



 Jun 12 2020



 next sibling parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 6/12/2020 5:17 PM, Andrei Alexandrescu wrote:

 Not sure about that part - if linkage was static by means of using the
"static" 
 keyword, multiple definitions may not be merged. (I may be wrong, please
correct 
 me.) Consider:
 
 static inline int fun() {
      static int x;
      return ++x;
 }
 
 In C++, each translation unit containing a definition of fun() will have a 
 distinct address for x. I don't see how the bodies of those functions can be 
 merged.


They are not merged in D, for the simple reason that ModuleA.fun() and 
ModuleB.fun() will have different (mangled) names presented to the linker.

(Of course, if you mark them extern(C) or extern(C++) the mangling will be the 
same and so they will be merged.)

As I've mentioned previously, C++ has all kinds of contortions trying to make 
inline work in a universe with:

1. .h file inclusion
2. no modules
3. no COMDATs

D just defines these problems out of existence.



 Jun 12 2020



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 6/12/20 8:54 PM, Walter Bright wrote:

 On 6/12/2020 5:17 PM, Andrei Alexandrescu wrote:

 Not sure about that part - if linkage was static by means of using the 
 "static" keyword, multiple definitions may not be merged. (I may be 
 wrong, please correct me.) Consider:

 static inline int fun() {
      static int x;
      return ++x;
 }

 In C++, each translation unit containing a definition of fun() will 
 have a distinct address for x. I don't see how the bodies of those 
 functions can be merged.


 
 They are not merged in D, for the simple reason that ModuleA.fun() and 
 ModuleB.fun() will have different (mangled) names presented to the linker.


For D the question is if they are merged if the function is defined in a 
.di file and imported in two other modules.



 Jun 12 2020



 next sibling parent  Manu <turkeyman gmail.com>  writes:


On Sat, Jun 13, 2020 at 1:30 PM Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 6/12/20 8:54 PM, Walter Bright wrote:

 On 6/12/2020 5:17 PM, Andrei Alexandrescu wrote:

 Not sure about that part - if linkage was static by means of using the
 "static" keyword, multiple definitions may not be merged. (I may be
 wrong, please correct me.) Consider:

 static inline int fun() {
      static int x;
      return ++x;
 }

 In C++, each translation unit containing a definition of fun() will
 have a distinct address for x. I don't see how the bodies of those
 functions can be merged.


 They are not merged in D, for the simple reason that ModuleA.fun() and
 ModuleB.fun() will have different (mangled) names presented to the


 linker.

 For D the question is if they are merged if the function is defined in a
 .di file and imported in two other modules.


They are not 'merged', they just don't exist.
The problem I've repeated many times for D is that it doesn't emit the
function ANYWHERE, and as such, you get a "undefined symbol" error. This is
different than C++ where you would have gotten a "multiply defined symbol"
error, but it's exactly the same problem for the exact same reason. It just
manifests differently because C++ has .h files which naturally duplicates
the code into each CU and D doesn't.



 Jun 12 2020



prev sibling next sibling parent  Manu <turkeyman gmail.com>  writes:


On Sat, Jun 13, 2020 at 1:39 PM Manu <turkeyman gmail.com> wrote:


 On Sat, Jun 13, 2020 at 1:30 PM Andrei Alexandrescu via Digitalmars-d <
 digitalmars-d puremagic.com> wrote:


 On 6/12/20 8:54 PM, Walter Bright wrote:

 On 6/12/2020 5:17 PM, Andrei Alexandrescu wrote:

 Not sure about that part - if linkage was static by means of using the
 "static" keyword, multiple definitions may not be merged. (I may be
 wrong, please correct me.) Consider:

 static inline int fun() {
      static int x;
      return ++x;
 }

 In C++, each translation unit containing a definition of fun() will
 have a distinct address for x. I don't see how the bodies of those
 functions can be merged.


 They are not merged in D, for the simple reason that ModuleA.fun() and
 ModuleB.fun() will have different (mangled) names presented to the


 linker.

 For D the question is if they are merged if the function is defined in a
 .di file and imported in two other modules.


 They are not 'merged', they just don't exist.
 The problem I've repeated many times for D is that it doesn't emit the
 function ANYWHERE, and as such, you get a "undefined symbol" error. This is
 different than C++ where you would have gotten a "multiply defined symbol"
 error, but it's exactly the same problem for the exact same reason. It just
 manifests differently because C++ has .h files which naturally duplicates
 the code into each CU and D doesn't.


And where I say "it's the same problem", perhaps it's better to say "it's
the same issue"; about the requirement to emit inline code to each
referencing CU, and what the linkonce link flags are designed to address.



 Jun 12 2020



prev sibling  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 6/12/2020 8:25 PM, Andrei Alexandrescu wrote:

 On 6/12/20 8:54 PM, Walter Bright wrote:

 On 6/12/2020 5:17 PM, Andrei Alexandrescu wrote:

 Not sure about that part - if linkage was static by means of using the 
 "static" keyword, multiple definitions may not be merged. (I may be wrong, 
 please correct me.) Consider:

 static inline int fun() {
      static int x;
      return ++x;
 }

 In C++, each translation unit containing a definition of fun() will have a 
 distinct address for x. I don't see how the bodies of those functions can be 
 merged.


 They are not merged in D, for the simple reason that ModuleA.fun() and 
 ModuleB.fun() will have different (mangled) names presented to the linker.


 
 For D the question is if they are merged if the function is defined in a .di 
 file and imported in two other modules.


If the di file is mentioned on the command line to the compiler, yes (1) 
instance of it appears in the executable. Otherwise, (0) instances of it appear 
in the executable. There are never 2 or more instances in the executable.



 Jun 12 2020



  parent reply Manu <turkeyman gmail.com>  writes:


On Sat, Jun 13, 2020 at 2:05 PM Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 6/12/2020 8:25 PM, Andrei Alexandrescu wrote:

 On 6/12/20 8:54 PM, Walter Bright wrote:

 On 6/12/2020 5:17 PM, Andrei Alexandrescu wrote:

 Not sure about that part - if linkage was static by means of using the
 "static" keyword, multiple definitions may not be merged. (I may be






 wrong,

 please correct me.) Consider:

 static inline int fun() {
      static int x;
      return ++x;
 }

 In C++, each translation unit containing a definition of fun() will






 have a

 distinct address for x. I don't see how the bodies of those functions






 can be

 merged.


 They are not merged in D, for the simple reason that ModuleA.fun() and
 ModuleB.fun() will have different (mangled) names presented to the




 linker.

 For D the question is if they are merged if the function is defined in a


 .di

 file and imported in two other modules.


 If the di file is mentioned on the command line to the compiler



It's not, that's literally the point of a .di file.



 , yes (1)
 instance of it appears in the executable. Otherwise, (0) instances of it
 appear
 in the executable. There are never 2 or more instances in the executable.


Exactly. And this is not a useful design.



 Jun 12 2020



 next sibling parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 6/12/2020 9:08 PM, Manu wrote:

     If the di file is mentioned on the command line to the compiler
 
 It's not, that's literally the point of a .di file.


No, it isn't. A .di file is more of a convention than a feature. It's a module 
and does not get special treatment by the compiler.


     , yes (1)
     instance of it appears in the executable. Otherwise, (0) instances of it
appear
     in the executable. There are never 2 or more instances in the executable.
 
 Exactly. And this is not a useful design.


I hate to say it, but these sorts of replies are completely useless to
resolving 
your issues. You omitted the *why*.

Why can't you put it on the command line?

May I suggest a paradigm shift? A .d (or .di) file is not a .h file and does
not 
work like .h files other than in a very superficial way. (In fact, .h files do 
not work like .h files. The C++ compiler has no idea what a .h file is. The 
characteristics you've imputed to it do not exist.)

It's a module. It's fundamentally different. The .h file rules you are used to 
do not apply.

You'd never put a .h file on the command line to a C++ compiler. Agreed. But 
those well-traveled notions don't apply to modules.



 Jun 13 2020



  parent reply Johannes Pfau <nospam example.com>  writes:


Am Sat, 13 Jun 2020 01:33:34 -0700 schrieb Walter Bright:


 On 6/12/2020 9:08 PM, Manu wrote:

     If the di file is mentioned on the command line to the compiler
 
 It's not, that's literally the point of a .di file.


 
 No, it isn't. A .di file is more of a convention than a feature. It's a
 module and does not get special treatment by the compiler.
 

     , yes (1)
     instance of it appears in the executable. Otherwise, (0) instances
     of it appear in the executable. There are never 2 or more instances
     in the executable.
 
 Exactly. And this is not a useful design.


 
 I hate to say it, but these sorts of replies are completely useless to
 resolving your issues. You omitted the *why*.
 
 Why can't you put it on the command line?
 



a.di:
void foo() {}

b.d:
import a;

c.d:
import a;
void main() {foo();}


dmd -c b.d
dmd -c c.d
dmd b.o c.o => undefined reference to `_D1a3fooFZv'

dmd -c a.di b.d -ofb.o
dmd -c a.di c.d -ofc.o
dmd b.o c.o => undefined reference to `_D1a3fooFZv'

mv a.di a.d
dmd -c a.d b.d -ofb.o
dmd -c a.d c.d -ofc.o
dmd b.o c.o => multiple definition of `_D1a12__ModuleInfoZ'

dmd -c a.d b.d -ofb.o -betterC
dmd -c a.d c.d -ofc.o -betterC
dmd b.o c.o => OK

BTW: If you do dmd -c a.di you get no object file output. So .di files 
are treated differently


I think it's interesting that DMD seems to emit some (all?) normal 
functions as weak. Not sure if LDC and GDC do the same thing. Would also 
be interesting to see how all this interacts with staic & shared 
libraries, though I'm optimistic that it just works.




So basically all that's missing for Manu's inline case would be to emit 
pragma(inlne) functions from non-root modules. Probably a 1-line change.
-- 
Johannes



 Jun 13 2020



 next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 6/13/20 7:31 AM, Johannes Pfau wrote:

 Am Sat, 13 Jun 2020 01:33:34 -0700 schrieb Walter Bright:
 

 On 6/12/2020 9:08 PM, Manu wrote:

      If the di file is mentioned on the command line to the compiler

 It's not, that's literally the point of a .di file.


 No, it isn't. A .di file is more of a convention than a feature. It's a
 module and does not get special treatment by the compiler.


      , yes (1)
      instance of it appears in the executable. Otherwise, (0) instances
      of it appear in the executable. There are never 2 or more instances
      in the executable.

 Exactly. And this is not a useful design.


 I hate to say it, but these sorts of replies are completely useless to
 resolving your issues. You omitted the *why*.

 Why can't you put it on the command line?


 
 
 a.di:
 void foo() {}
 
 b.d:
 import a;
 
 c.d:
 import a;
 void main() {foo();}
 
 
 dmd -c b.d
 dmd -c c.d
 dmd b.o c.o => undefined reference to `_D1a3fooFZv'
 
 dmd -c a.di b.d -ofb.o
 dmd -c a.di c.d -ofc.o
 dmd b.o c.o => undefined reference to `_D1a3fooFZv'
 
 mv a.di a.d
 dmd -c a.d b.d -ofb.o
 dmd -c a.d c.d -ofc.o
 dmd b.o c.o => multiple definition of `_D1a12__ModuleInfoZ'
 
 dmd -c a.d b.d -ofb.o -betterC
 dmd -c a.d c.d -ofc.o -betterC
 dmd b.o c.o => OK
 
 BTW: If you do dmd -c a.di you get no object file output. So .di files
 are treated differently
 
 
 I think it's interesting that DMD seems to emit some (all?) normal
 functions as weak. Not sure if LDC and GDC do the same thing. Would also
 be interesting to see how all this interacts with staic & shared
 libraries, though I'm optimistic that it just works.
 
 
 
 
 So basically all that's missing for Manu's inline case would be to emit
 pragma(inlne) functions from non-root modules. Probably a 1-line change.


Should that go in bugzilla?



 Jun 13 2020



  parent  Iain Buclaw <ibuclaw gdcproject.org>  writes:


On Saturday, 13 June 2020 at 18:24:18 UTC, Andrei Alexandrescu 
wrote:

 On 6/13/20 7:31 AM, Johannes Pfau wrote:

 
 So basically all that's missing for Manu's inline case would 
 be to emit
 pragma(inlne) functions from non-root modules. Probably a 
 1-line change.


 Should that go in bugzilla?


I don't see why there should not be a bugzilla for it.

 Manu, pardon my skipping the 11 pages of replies between the 
first and last posts, so if it's already been said, please ignore.

I think trying to impose the C/C++ notion of inline as a linkage 
modifier into D is at odds with modules.  I'm going to make a 
potentially disagreeable statement and say that there is no 
external or internal linkage in D.  Rather there's language 
linkage, e.g: `extern(C++)`; and then there's module linkage, 
where a given declaration resolves to the same symbol across all 
TUs that import its residing module, but otherwise not strictly 
visible outside that.

To address your laments, it may be wise to pay attention to the 
direction that `inline` is heading in C++ modules.

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1498r1.html#inline
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p1779r3.html

Notably the sentence: "Within interface units of modules, we 
suggest converging on an interperation more firmly rooted in the 
semantics: an inline entity (function definition or variable 
initialization) is semantically incorporated into (or inlined 
into) the interface of a module."

Which I interpret as, given:

     module mod;
     pragma(inline, true) void fun() { ... }

Any module that imports `mod` (called non-root),  will have the 
`fun` emitted into the module as well.  This would be implemented 
using the support of weak (ELF) or selectany (COFF), so the 
linker will throw away all duplicated definitions.

However, to satisfy your "emit to the binary only when it is 
called" constraint, it will also have to be discard-able at 
compile-time.  But I'm not sure whether that really a desirable 
feature to have.  At the very least, it should only extend to 
non-root modules, so there's always one copy in the root module 
it was declared, to avoid linker errors.



 Jun 21 2020



prev sibling  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 6/13/2020 4:31 AM, Johannes Pfau wrote:

 BTW: If you do dmd -c a.di you get no object file output. So .di files
 are treated differently


Yeah, you're right. Only in one spot, however, the .obj file is omitted. You
can 
always rename it to .d



 I think it's interesting that DMD seems to emit some (all?) normal
 functions as weak.


It emits them all as COMDATs. This is so multiple definitions can be combined.


 So basically all that's missing for Manu's inline case would be to emit
 pragma(inlne) functions from non-root modules. Probably a 1-line change.


It's simply unnecessary for what Manu wants to achieve (header only libraries).



 Jun 13 2020



  parent  Manu <turkeyman gmail.com>  writes:


On Sun, Jun 14, 2020 at 5:55 AM Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 6/13/2020 4:31 AM, Johannes Pfau wrote:

 BTW: If you do dmd -c a.di you get no object file output. So .di files
 are treated differently


 Yeah, you're right. Only in one spot, however, the .obj file is omitted.
 You can
 always rename it to .d


By definition; we're talking about libraries here. Inline's are only
relevant when calling into foreign code.
I don't molest the source tree of my libraries. There's a high probability
the directory doesn't even have write permissions.



 Jun 13 2020



prev sibling  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 6/13/20 12:08 AM, Manu wrote:

 On Sat, Jun 13, 2020 at 2:05 PM Walter Bright via Digitalmars-d 
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>> wrote:
 
     On 6/12/2020 8:25 PM, Andrei Alexandrescu wrote:
      > On 6/12/20 8:54 PM, Walter Bright wrote:
      >> On 6/12/2020 5:17 PM, Andrei Alexandrescu wrote:
      >>> Not sure about that part - if linkage was static by means of
     using the
      >>> "static" keyword, multiple definitions may not be merged. (I
     may be wrong,
      >>> please correct me.) Consider:
      >>>
      >>> static inline int fun() {
      >>>      static int x;
      >>>      return ++x;
      >>> }
      >>>
      >>> In C++, each translation unit containing a definition of fun()
     will have a
      >>> distinct address for x. I don't see how the bodies of those
     functions can be
      >>> merged.
      >>
      >> They are not merged in D, for the simple reason that
     ModuleA.fun() and
      >> ModuleB.fun() will have different (mangled) names presented to
     the linker.
      >
      > For D the question is if they are merged if the function is
     defined in a .di
      > file and imported in two other modules.
 
     If the di file is mentioned on the command line to the compiler
 
 
 It's not, that's literally the point of a .di file.
 
     , yes (1)
     instance of it appears in the executable. Otherwise, (0) instances
     of it appear
     in the executable. There are never 2 or more instances in the
     executable.
 
 
 Exactly. And this is not a useful design.


All I can say is if that and/or other wrinkles prevent header-only 
libraries definition and use, yes, not a useful design.

So Manu, is the top-level goal "I want header-only libraries in D"?



 Jun 13 2020



  parent  Manu <turkeyman gmail.com>  writes:


On Sun, Jun 14, 2020 at 4:20 AM Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 6/13/20 12:08 AM, Manu wrote:

 On Sat, Jun 13, 2020 at 2:05 PM Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>>


 wrote:

     On 6/12/2020 8:25 PM, Andrei Alexandrescu wrote:
      > On 6/12/20 8:54 PM, Walter Bright wrote:
      >> On 6/12/2020 5:17 PM, Andrei Alexandrescu wrote:
      >>> Not sure about that part - if linkage was static by means of
     using the
      >>> "static" keyword, multiple definitions may not be merged. (I
     may be wrong,
      >>> please correct me.) Consider:
      >>>
      >>> static inline int fun() {
      >>>      static int x;
      >>>      return ++x;
      >>> }
      >>>
      >>> In C++, each translation unit containing a definition of fun()
     will have a
      >>> distinct address for x. I don't see how the bodies of those
     functions can be
      >>> merged.
      >>
      >> They are not merged in D, for the simple reason that
     ModuleA.fun() and
      >> ModuleB.fun() will have different (mangled) names presented to
     the linker.
      >
      > For D the question is if they are merged if the function is
     defined in a .di
      > file and imported in two other modules.

     If the di file is mentioned on the command line to the compiler


 It's not, that's literally the point of a .di file.

     , yes (1)
     instance of it appears in the executable. Otherwise, (0) instances
     of it appear
     in the executable. There are never 2 or more instances in the
     executable.


 Exactly. And this is not a useful design.


 All I can say is if that and/or other wrinkles prevent header-only
 libraries definition and use, yes, not a useful design.

 So Manu, is the top-level goal "I want header-only libraries in D"?


It is one among multiple use cases in that category. It's essentially the

That is one useful case. It's not limited to that though, it comes up in
various situations.

For instance, I do a lot with DLL's... it's typical to load a DLL, and then
bind to a single function that creates an instance of some object that the
DLL provides. From there, further calls into the library are via the object
vtable.

From this point, there are a few design pressures that lead to inline


functions:
1. v-calls are slow, and trivial operations (ie, getters/setters) should
just be inlines where it makes sense. Cross-module calls are not desirable.
2. DLL's ABI is essentially part of the API, and changes to the ABI are
breaking changes to the lib.
  - Skilled design is to have fewer virtual functions that 'do stuff', with
flexibility in how it's called and lots of soft-extensibility option
arguments.
    * Enhancement to a lib can be made by adding a new item to an enum,
rather than breaking the ABI.
  - This broad style of function is inconvenient for users, and typical use
cases often wrap these work functions up in convenience inline calls, to
automate a complex calling scheme.
3. Such libraries are installed by the package manager in some location,
along with their 'headers', which root path is automatically added in a
build script somewhere.
  - It is not the job of the application's build environment to determine
what functions it calls from a lib and then cherry-pick source files from
the lib and include them into the applications build... that creates
compile-time couplings which are awkward at best, and naturally break-down
over time. It self-defeats the use of DLL's.

A build system might not even know how to identify the paths to a
particular library's random source files, and it shouldn't be the client's
responsibility to understand a lib's source tree. It's hard to know what
source files you need to include and build... I probably don't call every
function in the lib; I don't want to include the source for a DLL into my
application; that's self-defeating.

TL;DR, right now, I care about DLL's, and inline functions are an essential
part of DLL-based libraries.

It doesn't end there though either; there are so many cases where a
reference to a lib should not create a 'hard' coupling between projects. I
have often had situations involving 'optional' functionality; if a call is
never made, then the dependency beneath that call isn't inferred on the
client. An inline with no references naturally doesn't pull the dependency
reference into the binary unless it's actually called. You could argue for
stripping to solve that case, but that's a much more brittle and
unsatisfying solution.



 Jun 13 2020



prev sibling  parent reply Manu <turkeyman gmail.com>  writes:


On Sat, Jun 13, 2020 at 10:20 AM Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 6/12/20 2:08 PM, Johan wrote:

 On Friday, 12 June 2020 at 13:22:35 UTC, Andrei Alexandrescu wrote:

 On 6/8/20 2:14 AM, Manu wrote:

 In C/C++, inline says that a function will be emit to the binary only
 when it is called, and the function is marked with internal linkage
 (it is not visible to the linker from the symbol table)


 By my recollection this is not the case for C++, at all.

 * "inline" does NOT change a function's linkage in C++. You may have
 inline functions with internal linkage (static inline) or (default)
 external linkage. This is important because e.g. defining a static
 variable in an extern inline function will have the same address in
 all calls to the function.


 I believe Manu tried to explain that `inline` in C++ really only affects
 how the linker must treat the symbol, and it is best to remember that it
 does nothing at all concerning "inlining" (putting function body inside
 another).


 Thank you. That's not at all what he wrote. AT ALL. It's what I wrote.

 I'm emphasizing this because it has been a recurring problem: a
 legitimate problem with an explanation lost in translation.



You're joking right?
"""
1. *I only want the function to be present in the CALLING binary*. I do not
want an inline function present in the local binary where it was defined
(unless it was called internally). I do not want a linker to see the inline
function symbols and be able to link to them externally. [*This is about
linkage and controlling the binary or distribution environment*]
"""
There is absolutely nowhere in that text where I describe that a function
should be emit inline to the calling function. I couldn't possibly be
clearer about its placement into the calling CU.

I genuinely don't have any idea how I could have been misunderstood in this
thread. It seems most readers did not misunderstand me.


 `inline` is really a misnomer.

 The "linkage" effect of `inline` is required for C++ to work with
 #include files. There is some room for confusion because, depending on
 who is talking, the word "linkage" can have more nuances than just
 internal/external.


 It helps to use terminology that is well defined and standardized.
 That's where simply using the meaning defined and consistently used in
 the C++ standard document can be of great help.


We *are* using standard terminology. `inline` means something very specific
in C; it means deduplication of symbols, and that's what we're talking
about, for years.


 I personally take the broad stance of "linkage"

 including anything the linker can do with the symbol.


 Makes sense. It's really nice to just use
 http://eel.is/c++draft/basic.link, which ensures we all understand one
 another.


There's no text in there that describes the link flags with standard
terminology. I have never heard standard terminology for them.
Can you show a reference for standard linker terminology which we can refer
to in the future?

LLVM is the only reference I know:
https://llvm.org/docs/LangRef.html#linkage-types

`internal` or `linkonce` are possible choices here, `linkonce` would match
C++.


 Without `inline`, you would get multiple definition linker errors for

 functions defined in a header file when linking together two translation
 units that both included that header file. Instead of normal function
 linkage that forbids multiple definition, `inline` changes linkage to
 merge multiple definitions into one.


 Not sure about that part - if linkage was static by means of using the
 "static" keyword, multiple definitions may not be merged. (I may be
 wrong, please correct me.) Consider:

 static inline int fun() {
      static int x;
      return ++x;
 }

 In C++, each translation unit containing a definition of fun() will have
 a distinct address for x. I don't see how the bodies of those functions
 can be merged.


`static inline` is strictly internal.
They will not be merged. Each CU will have its own copy of the function,
and it will not deduplicate.

We should be able use `private` the same way in D.


 In addition, emission of the function must happen in any translation

 unit that references it (calling or address taken), and thus that
 translation unit must also define it (in contrast to just declaring it).
 I do not think `inline` forbids emission if the function is not


 referenced.

 I also think that's the case. (Also not what was claimed.)



have possibly been clearer about this. I've been saying it over and over
and over again for years.

The only difference you could perceive is that I suggest it should not emit
it if it's never referenced... to be clear; I was not commenting on what
C++ does, I was commenting on what I'd like.
It would be nice to not generate and emit the code and bloat the binary if
there are no calls. It's not necessary, because any external references
will have their own copy anyway.



 Jun 12 2020



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 6/12/20 9:13 PM, Manu wrote:

 On Sat, Jun 13, 2020 at 10:20 AM Andrei Alexandrescu via Digitalmars-d 
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>> wrote:
 
     On 6/12/20 2:08 PM, Johan wrote:
      > On Friday, 12 June 2020 at 13:22:35 UTC, Andrei Alexandrescu wrote:
      >> On 6/8/20 2:14 AM, Manu wrote:
      >>> In C/C++, inline says that a function will be emit to the
     binary only
      >>> when it is called, and the function is marked with internal
     linkage
      >>> (it is not visible to the linker from the symbol table)
      >>
      >> By my recollection this is not the case for C++, at all.
      >>
      >> * "inline" does NOT change a function's linkage in C++. You may
     have
      >> inline functions with internal linkage (static inline) or (default)
      >> external linkage. This is important because e.g. defining a static
      >> variable in an extern inline function will have the same address in
      >> all calls to the function.
      >
      > I believe Manu tried to explain that `inline` in C++ really only
     affects
      > how the linker must treat the symbol, and it is best to remember
     that it
      > does nothing at all concerning "inlining" (putting function body
     inside
      > another).
 
     Thank you. That's not at all what he wrote. AT ALL. It's what I wrote.
 
     I'm emphasizing this because it has been a recurring problem: a
     legitimate problem with an explanation lost in translation.
 
 
 You're joking right?


Well, again I tried to get communication going and again I failed. So... 
again I will bow out, with apologies.



 Jun 12 2020



  parent  Manu <turkeyman gmail.com>  writes:


On Sat, Jun 13, 2020 at 1:35 PM Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 6/12/20 9:13 PM, Manu wrote:

 On Sat, Jun 13, 2020 at 10:20 AM Andrei Alexandrescu via Digitalmars-d
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>>


 wrote:

     On 6/12/20 2:08 PM, Johan wrote:
      > On Friday, 12 June 2020 at 13:22:35 UTC, Andrei Alexandrescu


 wrote:

      >> On 6/8/20 2:14 AM, Manu wrote:
      >>> In C/C++, inline says that a function will be emit to the
     binary only
      >>> when it is called, and the function is marked with internal
     linkage
      >>> (it is not visible to the linker from the symbol table)
      >>
      >> By my recollection this is not the case for C++, at all.
      >>
      >> * "inline" does NOT change a function's linkage in C++. You may
     have
      >> inline functions with internal linkage (static inline) or


 (default)

      >> external linkage. This is important because e.g. defining a


 static

      >> variable in an extern inline function will have the same address


 in

      >> all calls to the function.
      >
      > I believe Manu tried to explain that `inline` in C++ really only
     affects
      > how the linker must treat the symbol, and it is best to remember
     that it
      > does nothing at all concerning "inlining" (putting function body
     inside
      > another).

     Thank you. That's not at all what he wrote. AT ALL. It's what I


 wrote.

     I'm emphasizing this because it has been a recurring problem: a
     legitimate problem with an explanation lost in translation.


 You're joking right?


 Well, again I tried to get communication going and again I failed. So...
 again I will bow out, with apologies.


No, I appreciate your effort to improve the conversation.
What I don't appreciate is suggesting that "That's not at all what [he]
wrote. AT ALL." when I really feel like it is exactly what I wrote.
And if it was confused, I apologise, but there are plenty of people who
were able to read it and understand precisely what I was saying.
Anyway...



 Jun 12 2020



prev sibling  parent reply Manu <turkeyman gmail.com>  writes:


On Fri, Jun 12, 2020 at 11:25 PM Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 6/8/20 2:14 AM, Manu wrote:

 In C/C++, inline says that a function will be emit to the binary only
 when it is called, and the function is marked with internal linkage (it
 is not visible to the linker from the symbol table)


 By my recollection this is not the case for C++, at all.

 * "inline" does NOT change a function's linkage in C++. You may have
 inline functions with internal linkage (static inline) or (default)
 external linkage. This is important because e.g. defining a static
 variable in an extern inline function will have the same address in all
 calls to the function.



It absolutely changes the linkage.
I believe it uses what LLVM calls 'ChooseOne' in its code generator, I
don't know about 'standard' linker terminology, if such a thing exists.

It's clearly in the spec too:
"""

   1. There may be more than one definition
   <https://en.cppreference.com/w/cpp/language/definition#One_Definition_Rule>
of
   an inline function or variable in the program as long as each
definition *appears
   in a different translation unit* and (for non-static inline functions and
   variables) all definitions are identical. For example, an inline
   function or an inline variable may be defined in a header file that is
   #include'd in multiple source files.

"""

If not for that link flag, you receive a "multiply declared symbol" error.

If the above is correct, a D language feature dedicated to inlining

 should do the following:

 * Always emit the function body in the .di file if ever asked to
 generate it.


Your sentence is unclear, but I think we agree.
I like how I describe this better: "The function must be emit to the
_calling_ CU"

* Never complain about duplicate symbols if an inline function has

 duplicate definitions.



And as I have been saying: "It must have the appropriate linker flag"
(again, I think LLVM calls it "ChooseOne")

Then compilers can decide on specific inlining strategies using the

 language feature as a hint.



useful to have SOME WAY, to 'force inline' and receive an error if it
failed.

performed in the front-end as some sort of AST macro, then we can have
complete confidence it happened, and error if not.
This is a DIFFERENT THING than 'inline' generally, and I would describe it
as "force" inline, and suggested `pragma(inline, force)`.

D does ALL of the above already for templates, so this is not a

 difficult feature to implement. In fact we use this technique (e.g. in
 druntime) by spelling inline as "()". Consider:

 int func(int) { ... body ... }

 Let's make this inline:

 int func()(int) { ... body ... }

 Done.


Except for the sea of edge cases when you supply `func` to generic code
that was expecting a function.
This is a sloppy and unsatisfying work-around.

In D there's one additional implication of body availability - the

 function is eligible for CTFE. I think any adjustment to the extant
 inline pragma needs to make this a top-level consideration.


I think this will work naturally with no changes today. If a function
definition is present in a di file, you can CTFE call it, but (unless we
fix inline) you can not call it at runtime (because no code will be
emit anywhere).



 Jun 12 2020



 next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 6/12/20 8:52 PM, Manu wrote:

 On Fri, Jun 12, 2020 at 11:25 PM Andrei Alexandrescu via Digitalmars-d 
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>> wrote:
 
     On 6/8/20 2:14 AM, Manu wrote:
      > In C/C++, inline says that a function will be emit to the binary
     only
      > when it is called, and the function is marked with internal
     linkage (it
      > is not visible to the linker from the symbol table)
 
     By my recollection this is not the case for C++, at all.
 
     * "inline" does NOT change a function's linkage in C++. You may have
     inline functions with internal linkage (static inline) or (default)
     external linkage. This is important because e.g. defining a static
     variable in an extern inline function will have the same address in all
     calls to the function.
 
 
 It absolutely changes the linkage.


No.


 I believe it uses what LLVM calls 'ChooseOne' in its code generator, I 
 don't know about 'standard' linker terminology, if such a thing exists.


It does: http://eel.is/c++draft/basic.link


 It's clearly in the spec too:
 """
 
  1. There may be more than one definition
     <https://en.cppreference.com/w/cpp/language/definition#One_Definition_Rule> of
     an inline function or variable in the program as long as each
     definition *appears in a different translation unit* and (for
     non-static inline functions and variables) all definitions are
     identical. For example, an inline function or an inline variable may
     be defined in a header file that is #include'd in multiple source files.
 
 """


The quote does not even contain the word "linkage".

I'm insisting on this because it happens so often. We need to use the 
terms with the same meaning, otherwise we get bogged down in silly side 
quests "it doesn't change linkage" - "oh but it does" and there is no 
progress.



 Jun 12 2020



  parent  Manu <turkeyman gmail.com>  writes:


On Sat, Jun 13, 2020 at 1:25 PM Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 6/12/20 8:52 PM, Manu wrote:

 On Fri, Jun 12, 2020 at 11:25 PM Andrei Alexandrescu via Digitalmars-d
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>>


 wrote:

     On 6/8/20 2:14 AM, Manu wrote:
      > In C/C++, inline says that a function will be emit to the binary
     only
      > when it is called, and the function is marked with internal
     linkage (it
      > is not visible to the linker from the symbol table)

     By my recollection this is not the case for C++, at all.

     * "inline" does NOT change a function's linkage in C++. You may have
     inline functions with internal linkage (static inline) or (default)
     external linkage. This is important because e.g. defining a static
     variable in an extern inline function will have the same address in


 all

     calls to the function.


 It absolutely changes the linkage.


 No.


Dump a binary with and without `inline`; look at the link flags... they are
different.


 I believe it uses what LLVM calls 'ChooseOne' in its code generator, I

 don't know about 'standard' linker terminology, if such a thing exists.


 It does: http://eel.is/c++draft/basic.link


 It's clearly in the spec too:
 """

  1. There may be more than one definition
     <


 https://en.cppreference.com/w/cpp/language/definition#One_Definition_Rule

 of
     an inline function or variable in the program as long as each
     definition *appears in a different translation unit* and (for
     non-static inline functions and variables) all definitions are
     identical. For example, an inline function or an inline variable may
     be defined in a header file that is #include'd in multiple source


 files.

 """


 The quote does not even contain the word "linkage".

 I'm insisting on this because it happens so often. We need to use the
 terms with the same meaning, otherwise we get bogged down in silly side
 quests "it doesn't change linkage" - "oh but it does" and there is no
 progress.


That's because this is a C++ spec, and C++ is not a linker. The spec just
specifies the required semantics, and the compiler implements them.
The compiler happens to implement them appropriately for the linking
ecosystem (by using appropriate link flags), because that's how native code
works... you build modules and then link them with a linker which is
outside the domain of the language spec.

The whole point is that different languages can interact with each
other successfully in 'linker space'. Unless we want to have a
self-contained language island, we can't ignore that a linker will link our
code (together with other code), and the compilers output has to work
properly in that environment.
Like C++, we don't need to spec details about link flags, but the
compiler implementation still needs to implement them in those terms, cus
that's how the ecosystem has been designed to work.

I imagine any changes made here would have a similar expression in the D
spec as what you read in the C++ spec... and the practical reality is the
implementation will apply the appropriate link flags to the symbols.



 Jun 12 2020



prev sibling  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 6/12/20 8:52 PM, Manu wrote:


 to have SOME WAY, to 'force inline' and receive an error if it failed.


I recall a couple of compilers (TopSpeed, does anyone remember?) had 
such a feature. The warnings listing the arbitrary functions that failed 
whatever heuristics was utterly useless. No C++ compiler implements it 
today, and I don't think any should.



 Jun 12 2020



 next sibling parent  Manu <turkeyman gmail.com>  writes:


On Sat, Jun 13, 2020 at 1:25 PM Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 6/12/20 8:52 PM, Manu wrote:


 to have SOME WAY, to 'force inline' and receive an error if it failed.


 I recall a couple of compilers (TopSpeed, does anyone remember?) had
 such a feature. The warnings listing the arbitrary functions that failed
 whatever heuristics was utterly useless. No C++ compiler implements it
 today, and I don't think any should.


I feel like I clearly agreed here too; I gave 3 cases which are distinct

those cases.

agree C++ shouldn't try and jam this idea into `inline` because that has a
confused history, and a fairly well defined present.
That doesn't mean that it's not a useful tool though, and one that I've
wanted lots of times... but as I've made clear, I see this as a distinct
use case, and should be explicit and distinct.






common use cases.



 Jun 12 2020



prev sibling  parent  Johannes Pfau <nospam example.com>  writes:


Am Fri, 12 Jun 2020 23:24:32 -0400 schrieb Andrei Alexandrescu:


 On 6/12/20 8:52 PM, Manu wrote:


 to have SOME WAY, to 'force inline' and receive an error if it failed.


 
 I recall a couple of compilers (TopSpeed, does anyone remember?) had
 such a feature. The warnings listing the arbitrary functions that failed
 whatever heuristics was utterly useless. No C++ compiler implements it
 today, and I don't think any should.


Doesn't GCC's always_inline do exactly that?

https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html
Generally, functions are not inlined unless optimization is specified. 
For functions declared inline, this attribute inlines the function 
independent of any restrictions that otherwise apply to inlining. Failure 
to inline such a function is diagnosed as an error. Note that if such a 
function is called indirectly the compiler may or may not inline it 
depending on optimization level and a failure to inline an indirect call 
may or may not be diagnosed. 



-- 
Johannes



 Jun 13 2020