www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Discussion on static reflection syntax in C++

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Of possible interest:

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf
Feb 22 2021
next sibling parent reply 12345swordy <alexanderheistermann gmail.com> writes:
On Monday, 22 February 2021 at 16:27:49 UTC, Andrei Alexandrescu 
wrote:

 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf


Great, let's make c++ more confusing to newbies.

-Alex
Feb 22 2021
parent reply 12345swordy <alexanderheistermann gmail.com> writes:
On Monday, 22 February 2021 at 16:43:34 UTC, 12345swordy wrote:

 On Monday, 22 February 2021 at 16:27:49 UTC, Andrei 
 Alexandrescu wrote:

 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf


 Great, let's make c++ more confusing to newbies.

 -Alex


Oh great! The named parameter proposal for c++ is got to be the 
worst version of named parameters that I ever seen!
https://groups.google.com/a/isocpp.org/g/std-proposals/c/3dUkwyp2Ie4/m/rZ8dgxVlCgAJ

-Alex
Feb 22 2021
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 23/02/2021 7:22 AM, 12345swordy wrote:

 Oh great! The named parameter proposal for c++ is got to be the worst 
 version of named parameters that I ever seen!
 https://groups.google.com/a/isocpp.org/g/std-proposals/c/3dUkwyp2Ie4/m/rZ8dgxVlCgAJ


They really went with the kitchen sink approach didn't they?
Yikes.
Feb 22 2021
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Feb 23, 2021 at 12:35:31PM +1300, rikki cattermole via Digitalmars-d
wrote:

 On 23/02/2021 7:22 AM, 12345swordy wrote:

 Oh great! The named parameter proposal for c++ is got to be the worst
 version of named parameters that I ever seen!
 https://groups.google.com/a/isocpp.org/g/std-proposals/c/3dUkwyp2Ie4/m/rZ8dgxVlCgAJ


 
 They really went with the kitchen sink approach didn't they?
 Yikes.


Seeing as this is C++ we're talking about, a kitchen sink approach
doesn't surprise me. (Even if it does disgust me. :-D)


T

-- 
Life is too short to run proprietary software. -- Bdale Garbee
Feb 22 2021
prev sibling next sibling parent reply bachmeier <no spam.net> writes:
On Monday, 22 February 2021 at 16:27:49 UTC, Andrei Alexandrescu 
wrote:

 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf


Good this is proposed for C++ and not for D. The 
complexity/benefit ratio of adding another meaning of ^ and 
[:refl:] is rather high. Citing * and & as models for ^ is 
reasonable only to someone that has not tried to teach others to 
program. At least to me, this is horrible:

f<([:Refl:])>();

In complete seriousness, it would be better to use emoji than to 
write things like that.
Feb 22 2021
parent Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Monday, 22 February 2021 at 16:47:13 UTC, bachmeier wrote:

 On Monday, 22 February 2021 at 16:27:49 UTC, Andrei 
 Alexandrescu wrote:

 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf


 Good this is proposed for C++ and not for D. The 
 complexity/benefit ratio of adding another meaning of ^ and 
 [:refl:] is rather high. Citing * and & as models for ^ is 
 reasonable only to someone that has not tried to teach others 
 to program. At least to me, this is horrible:

 f<([:Refl:])>();


aka as
<:Rofl:> <º))))><


 In complete seriousness, it would be better to use emoji than 
 to write things like that.
Feb 23 2021
prev sibling next sibling parent reply IGotD- <nise nise.com> writes:
On Monday, 22 February 2021 at 16:27:49 UTC, Andrei Alexandrescu 
wrote:

 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf


Thank you for posting this. Oh my god, let's hope this goes 
through and let C++ destroy itself. It seems like the C++ 
maintainers are becoming more and more obsessed with colons.
Feb 22 2021
parent reply Dukc <ajieskola gmail.com> writes:
On Monday, 22 February 2021 at 16:54:31 UTC, IGotD- wrote:

 Oh my god, let's hope this goes through and let C++ destroy 
 itself.


I can't agree with this wish - not everybody can switch. If 
something like this goes through it'd only make life more 
difficult to those who don't have the luxury of D (or Rust, Go, 
Nim, etc.). I believe C++ is still an improvement over C (at 
least if you don't go crazy with all the complex features).

But I too kind of fail to see how `f(...[:tuple:]..., a, b)` 
would cut it when `tuple.expand.f(a,b)` has been possible in 
"another language" for like 10 years.
Feb 22 2021
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Feb 22, 2021 at 07:27:29PM +0000, Dukc via Digitalmars-d wrote:

 On Monday, 22 February 2021 at 16:54:31 UTC, IGotD- wrote:

 Oh my god, let's hope this goes through and let C++ destroy itself.


 
 I can't agree with this wish - not everybody can switch. If something
 like this goes through it'd only make life more difficult to those who
 don't have the luxury of D (or Rust, Go, Nim, etc.).


That would perhaps put more pressure on C++ devs to switch to a saner
language! ;-)



 I believe C++ is still an improvement over C (at least if you don't go
 crazy with all the complex features).


That is precisely the problem: all the complex features are still there,
beckoning every new hire to use them as a quick-fix to make a release
deadline.  Unless you keep a very tight control over what features can
be used -- essentially balkanizing the language, which has already
happened for at least a decade, probably longer -- the codebase becomes
a gigantic, unmaintainable mess.

Years ago, I worked in a team project where there was a C++-based
infrastructure so fancy and over-engineered, that after a while nobody
knew how to use it properly and started spending more time working
around it than using it.  Eventually, we ditched the C++ portion
completely and rewrote it from scratch in C.  It was a refreshing
change.  C's relative dearth of features was certainly limiting, but
limitation is not a bad thing in a team where people are constantly
coming and going. No matter how horribly the code devolved, there was
still only a small set of features it could abuse, and the scope of
abuse is well-known and manageable.  It wasn't *enjoyable* to go back to
C per se (D takes the cake on that one :-D), but it was definitely
better than drowning in the ocean of badly-interacting misfeatures that
is C++.



 But I too kind of fail to see how `f(...[:tuple:]..., a, b)` would cut it
 when `tuple.expand.f(a,b)` has been possible in "another language" for like
 10 years.


That's why C++'s early death would do everyone some good. Maybe some of
the survivors would discover D. :-D


T

-- 
Programming is not just an act of telling a computer what to do: it is also an
act of telling other programmers what you wished the computer to do. Both are
important, and the latter deserves care. -- Andrew Morton
Feb 22 2021
parent reply Dukc <ajieskola gmail.com> writes:
On Monday, 22 February 2021 at 21:16:07 UTC, H. S. Teoh wrote:

 That would perhaps put more pressure on C++ devs to switch to a 
 saner language! ;-)


But who would maintain the C++ programs then? There may be a 
reasonable case to discourage new projects in C++, but's that's 
hardly the same as trying to kill C++ programming.

Until we can automatically transpile everything I quess.



 Years ago, I worked in a team project where there was a 
 C++-based infrastructure so fancy and over-engineered, that 
 after a while nobody knew how to use it properly and started 
 spending more time working around it than using it.  
 Eventually, we ditched the C++ portion completely and rewrote 
 it from scratch in C.  It was a refreshing change.  C's 
 relative dearth of features was certainly limiting, but 
 limitation is not a bad thing in a team where people are 
 constantly coming and going. No matter how horribly the code 
 devolved, there was still only a small set of features it could 
 abuse, and the scope of abuse is well-known and manageable.  It 
 wasn't *enjoyable* to go back to C per se (D takes the cake on 
 that one :-D), but it was definitely better than drowning in 
 the ocean of badly-interacting misfeatures that is C++.


Hmm, I see. The engineer in me says it's foolish to ditch all 
C++, keep the function overloading and simple templates at least 
(I'm thinking something like `T binaryFun<T>(T arg1, T arg2)`). 
But I can kinda see that rules like that might have weaker 
authority than a simple language change. I'm still suspicious 
about the tradeoff you made but at least I see potential reason 
to accept it.


 But I too kind of fail to see how `f(...[:tuple:]..., a, b)` 
 would cut it
 when `tuple.expand.f(a,b)` has been possible in "another 
 language" for like
 10 years.


 That's why C++'s early death would do everyone some good. Maybe 
 some of the survivors would discover D. :-D


I meant that I don't see how it would cut to add as a new 
language feature. But for the hapless programmer who is down to 
either it, the C preprocessor, or copypasta, it just might make 
sense IMO. Just barely, and just maybe.
Feb 22 2021
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Feb 22, 2021 at 09:47:48PM +0000, Dukc via Digitalmars-d wrote:

 On Monday, 22 February 2021 at 21:16:07 UTC, H. S. Teoh wrote:

 That would perhaps put more pressure on C++ devs to switch to a
 saner language! ;-)


 
 But who would maintain the C++ programs then? There may be a
 reasonable case to discourage new projects in C++, but's that's hardly
 the same as trying to kill C++ programming.


Relax, nobody is "trying to kill C++ programming". But the more the C++
committee tries to add more fat to an already overweight beast of a
language, its inevitable demise draws nearer, and bystanders like myself
enjoy speculating on the benefits that everyone would enjoy once the
beast is finally, fully, dead.



 Until we can automatically transpile everything I quess.


I don't think you need to worry; the industry has enough C++ code around
that it will last for a *long* time yet.  And if the beast ever actually
dies, I'm sure more than just a handful of clever heads would come up
with a good way of migrating everything over. (More like, everyone and
their neighbour's dog will invent a way, of which may be a handful will
survive the test of real-world code and become widely disseminated as
the solution(s) to the migration problem.)



 Years ago, I worked in a team project where there was a C++-based
 infrastructure so fancy and over-engineered, that after a while
 nobody knew how to use it properly and started spending more time
 working around it than using it.  Eventually, we ditched the C++
 portion completely and rewrote it from scratch in C.  It was a
 refreshing change.  C's relative dearth of features was certainly
 limiting, but limitation is not a bad thing in a team where people
 are constantly coming and going. No matter how horribly the code
 devolved, there was still only a small set of features it could
 abuse, and the scope of abuse is well-known and manageable.  It
 wasn't *enjoyable* to go back to C per se (D takes the cake on that
 one :-D), but it was definitely better than drowning in the ocean of
 badly-interacting misfeatures that is C++.


 
 Hmm, I see. The engineer in me says it's foolish to ditch all C++,
 keep the function overloading and simple templates at least (I'm
 thinking something like `T binaryFun<T>(T arg1, T arg2)`).


Some of the programmers on the team were so badly burned by the
experience that they flinch at the mere sight of templates at a distance
of 10 feet away.



 But I can kinda see that rules like that might have weaker authority
 than a simple language change.


We're talking about real world code here, written in a real company by
real people (most of whom have moved on to better things) and running in
real customer environments.  Experience has shown time and again that
such "rules" do not work. Never did work. It's programming by
convention, as Walter often refers to, and everyone knows what happens
when the release deadline is looming and everyone has more on their
plate than they could possibly hope to finish.  You just reach out and
use whatever straw you could grasp to get it working and move on.  If a
feature is in the language, people will use it. The boss isn't going to
stop a release just because somebody broke the rules and as a result got
the code working by the deadline. (As opposed to, y'know, sticking to
the rules and *not* shipping on time.)

Besides, having features in the language that nobody ever uses is really
pointless. Better throw it out and end up with a better, slimmer
language.



 I'm still suspicious about the tradeoff you made but at least I see
 potential reason to accept it.


[...]

I wasn't the one who made the decision.  If it were up to me, I'd have
stuck with C++ somehow. But I can totally sympathize with why the PTB
chose to ditch it.  It's hard not to sympathize when I've had the
dubious privilege of writing a last-minute hack function call that
involved 6 layers of abstraction, one of which involves an RPC bridge
that uses fwrite, fread, and a temporary file in order to bridge the gap
between the C++ and non-C++ parts of the code. (You *know* something has
gone horribly wrong when the act of making a function call involves disk
I/O and a temporary file (a *hard-coded* one...). And that's on top of
several other layers of marshalling/unmarshalling.  Don't ask, you do
*not* want to know.)

The day we finally could delete the entire C++ branch of the source
there was a veritable celebration. We typed every `rm -rf` command with
relish and exquisite delight. :-P


T

-- 
Why do conspiracy theories always come from the same people??
Feb 22 2021
prev sibling next sibling parent reply SealabJaster <sealabjaster gmail.com> writes:
On Monday, 22 February 2021 at 16:27:49 UTC, Andrei Alexandrescu 
wrote:

 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf


"Readability. Obviously, we’d like our programs to be readable. 
We want syntax that is both visually distinctive yet 
comprehensible."

Can anyone more experienced with C++ confirm that this is in any 
way readable and easy to understand? Because my definition of 
"readability" appears to be vastly different, especially when I 
imagine it being used alongside the rest of C++'s symbol spam.

"f<([:Refl:])>();"

wat

"(: R :). Looks like smiley faces"

But apparently "[:" and ":]" don't. Part of me feels like the 
designer(s) really wanted to use square brackets for that extra 
*artistic touch*.

I don't really have anything constructive to say about this 
(presumably) draft proposal, because the actual feature itself is 
of course pretty useful, and I'm not really qualified to say 
whether this proposal is a good fit or not. But why is it that 
C++ uses such unusual syntax for everything?
Feb 22 2021
next sibling parent SealabJaster <sealabjaster gmail.com> writes:
On Monday, 22 February 2021 at 18:01:34 UTC, SealabJaster wrote:

 On Monday, 22 February 2021 at 16:27:49 UTC, Andrei 
 Alexandrescu wrote:

 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf




Also for that first example under 5.3.3, I hope that they made a 
mistake when writing it, because I really don't see how `cout << 
%x;` prints the value of "V" while `cout << t.%x;` prints "t.m". 
I ask this because I have a slight concern that this might 
actually be intentional.
Feb 22 2021
prev sibling parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Monday, 22 February 2021 at 18:01:34 UTC, SealabJaster wrote:

 On Monday, 22 February 2021 at 16:27:49 UTC, Andrei 
 Alexandrescu wrote:

 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf


 "Readability. Obviously, we’d like our programs to be readable. 
 We want syntax that is both visually distinctive yet 
 comprehensible."

 Can anyone more experienced with C++ confirm that this is in 
 any way readable and easy to understand? Because my definition 
 of "readability" appears to be vastly different, especially 
 when I imagine it being used alongside the rest of C++'s symbol 
 spam.

 "f<([:Refl:])>();"


🤣

Is this real life
Feb 23 2021
parent reply Guillaume Piolat <first.name spam.org> writes:
On Tuesday, 23 February 2021 at 10:14:50 UTC, Imperatorn wrote:

 Is this real life


C++ is quickly becoming that annoying legacy langage that no one 
likes.
I would be incredibly wary starting a C++ project in 2021 ; in my 
exprience literally anything else is a more productive use of 
your time.

Even in the audio space which is 95% C++, the top product is 
FLStudio, a product that was originally from Delphi.
Friendlier languages lead to friendlier products for some reasons.
Feb 23 2021
next sibling parent reply Paulo Pinto <pjmlp progtools.org> writes:
On Tuesday, 23 February 2021 at 10:24:26 UTC, Guillaume Piolat 
wrote:

 On Tuesday, 23 February 2021 at 10:14:50 UTC, Imperatorn wrote:

 Is this real life


 C++ is quickly becoming that annoying legacy langage that no 
 one likes.
 I would be incredibly wary starting a C++ project in 2021 ; in 
 my exprience literally anything else is a more productive use 
 of your time.

 Even in the audio space which is 95% C++, the top product is 
 FLStudio, a product that was originally from Delphi.
 Friendlier languages lead to friendlier products for some 
 reasons.


The effort to avoid C++ in some domains is very high, for example 
on Windows, it seems every effort from DevDiv to improve the 
overall experience gets botched by WinDev pushing their C++ 
agenda.

Now it remains to be seen what uphill battles the newly created 
Microsoft Rust team will need to face as well.

This is just one example among many, either one is prepared to be 
the snowflake that does everything alone, and spends development 
money getting stuff to work, or just integrate the existing 
ecosystem and proceed to spend customer's money on actual product 
development.

I cannot bill hours on "working around existing C++ SDK, because 
there is better out there".
Feb 23 2021
parent Guillaume Piolat <first.name spam.org> writes:
On Tuesday, 23 February 2021 at 13:31:00 UTC, Paulo Pinto wrote:

 spends development money getting stuff to work, or just 
 integrate the existing ecosystem and proceed to spend 
 customer's money on actual product development.


It costs very real development money to live with C++ and 
"getting stuff to work". If anything, D is much more 
debt-conscious since frontends are shared and you can use the 
LLVM target for about everything.
Feb 23 2021
prev sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 23 February 2021 at 10:24:26 UTC, Guillaume Piolat 
wrote:

 On Tuesday, 23 February 2021 at 10:14:50 UTC, Imperatorn wrote:

 Is this real life


 C++ is quickly becoming that annoying legacy langage that no 
 one likes.
 I would be incredibly wary starting a C++ project in 2021 ; in 
 my exprience literally anything else is a more productive use 
 of your time.


I agree that C++ is less productive than many other languages if 
you want to write the whole application in C++. It is quite ok 
for "engine parts", e.g. DSP etc.

But I don't think this syntax thing is going to be a big issue. 
Editors will colour them and it static reflection is most likely 
something library authors will use.


 Even in the audio space which is 95% C++, the top product is 
 FLStudio, a product that was originally from Delphi.


I though most people use Abelton Live? Although Cubase is 
probably better for composing than both Abelton and FLStudio.
Feb 23 2021
prev sibling next sibling parent reply Max Haughton <maxhaton gmail.com> writes:
On Monday, 22 February 2021 at 16:27:49 UTC, Andrei Alexandrescu 
wrote:

 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf


Exposing reflection information via CTFE seems like a much more 
workable solution - i.e. you just write regular D code. This is 
what Stefan's work effectively culminates in.
Feb 22 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 22 February 2021 at 21:34:47 UTC, Max Haughton wrote:

 On Monday, 22 February 2021 at 16:27:49 UTC, Andrei 
 Alexandrescu wrote:

 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf


 Exposing reflection information via CTFE seems like a much more 
 workable solution - i.e. you just write regular D code. This is 
 what Stefan's work effectively culminates in.


Or, in other words: procedural macros. (Shh, don't tell Walter.)
Feb 22 2021
parent reply Max Haughton <maxhaton gmail.com> writes:
On Monday, 22 February 2021 at 21:59:41 UTC, Paul Backus wrote:

 On Monday, 22 February 2021 at 21:34:47 UTC, Max Haughton wrote:

 On Monday, 22 February 2021 at 16:27:49 UTC, Andrei 
 Alexandrescu wrote:

 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf


 Exposing reflection information via CTFE seems like a much 
 more workable solution - i.e. you just write regular D code. 
 This is what Stefan's work effectively culminates in.


 Or, in other words: procedural macros. (Shh, don't tell Walter.)


They aren't macros though - at least in my mind a macro is 
effectively a shortcut into the bowels of the compiler whereas in 
this case the compiler wouldn't care what you do with the 
information after it gives it to you since it's just regular D 
code.
Feb 22 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 22 February 2021 at 23:32:39 UTC, Max Haughton wrote:

 On Monday, 22 February 2021 at 21:59:41 UTC, Paul Backus wrote:

 On Monday, 22 February 2021 at 21:34:47 UTC, Max Haughton 
 wrote:

 On Monday, 22 February 2021 at 16:27:49 UTC, Andrei 
 Alexandrescu wrote:

 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf


 Exposing reflection information via CTFE seems like a much 
 more workable solution - i.e. you just write regular D code. 
 This is what Stefan's work effectively culminates in.


 Or, in other words: procedural macros. (Shh, don't tell 
 Walter.)


 They aren't macros though - at least in my mind a macro is 
 effectively a shortcut into the bowels of the compiler whereas 
 in this case the compiler wouldn't care what you do with the 
 information after it gives it to you since it's just regular D 
 code.


Aren't macros in Lisp just regular Lisp code, too? :)

As far as I'm concerned, a procedural macro is a function that

1. runs at compile time,
2. takes AST nodes as input, and
3. produces AST nodes as output.

Type functions fit those criteria perfectly. Currently, they only 
work on a subset of AST nodes (types), so they're not a 
*complete* implementation of procedural macros, but generalize 
them enough and that's what you'll get.
Feb 22 2021
next sibling parent reply Bruce Carneal <bcarneal gmail.com> writes:
On Monday, 22 February 2021 at 23:44:45 UTC, Paul Backus wrote:

 As far as I'm concerned, a procedural macro is a function that

 1. runs at compile time,
 2. takes AST nodes as input, and
 3. produces AST nodes as output.

 Type functions fit those criteria perfectly. Currently, they 
 only work on a subset of AST nodes (types), so they're not a 
 *complete* implementation of procedural macros, but generalize 
 them enough and that's what you'll get.


Yes.

Given the collective experience that we've had with D's meta 
programming facilities it would be surprising if we could not 
imagine something significantly better at this point.  By 
"better" I mostly mean facilities that would allow us to write 
code that is more readable, more composable, more testable, and 
more easily debuggable than an equally performant solution using 
current practice.

I'm sympathetic to both type functions and MTVs (monadic type 
variables), CT type objects with actual memory footprint that 
function as compiler-guaranteed-safe cursors within the lazily 
realized dependent type space of a D program.

Beyond those two I know that Stefan, and from the above and 
earlier writings I expect Paul, have thought about more powerful 
yet still tractable capabilities.  It feels like we're within 
reach of a significant advance.
Feb 22 2021
parent Max Haughton <maxhaton gmail.com> writes:
On Tuesday, 23 February 2021 at 05:17:04 UTC, Bruce Carneal wrote:

 On Monday, 22 February 2021 at 23:44:45 UTC, Paul Backus wrote:

 As far as I'm concerned, a procedural macro is a function that

 1. runs at compile time,
 2. takes AST nodes as input, and
 3. produces AST nodes as output.

 Type functions fit those criteria perfectly. Currently, they 
 only work on a subset of AST nodes (types), so they're not a 
 *complete* implementation of procedural macros, but generalize 
 them enough and that's what you'll get.


 Yes.

 Given the collective experience that we've had with D's meta 
 programming facilities it would be surprising if we could not 
 imagine something significantly better at this point.  By 
 "better" I mostly mean facilities that would allow us to write 
 code that is more readable, more composable, more testable, and 
 more easily debuggable than an equally performant solution 
 using current practice.

 I'm sympathetic to both type functions and MTVs (monadic type 
 variables), CT type objects with actual memory footprint that 
 function as compiler-guaranteed-safe cursors within the lazily 
 realized dependent type space of a D program.

 Beyond those two I know that Stefan, and from the above and 
 earlier writings I expect Paul, have thought about more 
 powerful yet still tractable capabilities.  It feels like we're 
 within reach of a significant advance.


I have been talking to Stefan quite a lot about typefunctions 
recently, I'm reasonably convinced they are the most natural 
proposal to explore for D at the moment.

One thing I have been toying with is calling them metafunctions 
instead since they clearly don't only consider types in the 
general case.


Having a well defined type hierarchy to represent the AST in a 
manner seperate to the actual compiler has an added benefit of 
making it trivial to write clean programs that analyse D code 
without bringing in the entire compiler or going without sema.
Feb 22 2021
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 22 February 2021 at 23:44:45 UTC, Paul Backus wrote:

 On Monday, 22 February 2021 at 23:32:39 UTC, Max Haughton wrote:

 On Monday, 22 February 2021 at 21:59:41 UTC, Paul Backus wrote:

 On Monday, 22 February 2021 at 21:34:47 UTC, Max Haughton 
 wrote:

 On Monday, 22 February 2021 at 16:27:49 UTC, Andrei 
 Alexandrescu wrote:

 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf


 Exposing reflection information via CTFE seems like a much 
 more workable solution - i.e. you just write regular D code. 
 This is what Stefan's work effectively culminates in.


 Or, in other words: procedural macros. (Shh, don't tell 
 Walter.)


 They aren't macros though - at least in my mind a macro is 
 effectively a shortcut into the bowels of the compiler whereas 
 in this case the compiler wouldn't care what you do with the 
 information after it gives it to you since it's just regular D 
 code.


 Aren't macros in Lisp just regular Lisp code, too? :)

 As far as I'm concerned, a procedural macro is a function that

 1. runs at compile time,
 2. takes AST nodes as input, and
 3. produces AST nodes as output.

 Type functions fit those criteria perfectly. Currently, they 
 only work on a subset of AST nodes (types), so they're not a 
 *complete* implementation of procedural macros, but generalize 
 them enough and that's what you'll get.


They don't exactly take AST nodes.
They take a constrained (stabilized) datastructure which reflects 
a subset of the ast.
At least conceptually.
They don't create ast nodes, but regular values, which may be 
converted to ast-nodes in the case where they are statically 
available. (currently only types)

My currently exploration leads me to a path where you can 
_extend_ open scopes within typefunction by using special 
functions which map provide a stable mapping to compiler 
intrinsics.
such as __struct* __add_struct(string name, __symbol[] members);
the __struct* will not be a proper struct type unless
__type__ __add_to_scope(__scope Scope, __struct* s);
has been called.

At which point any modification of the __struct* is an error and 
will abort compilation.

I am still searching for a solution which is more declarative, 
but I fear for heavy usage (10_000 new types and declarations 
added) scenarios the procedural imperative style is just more 
optimizable.

I am always open to syntax suggestions, as I don't like the one I 
have :)
Feb 23 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 23 February 2021 at 14:14:13 UTC, Stefan Koch wrote:

 On Monday, 22 February 2021 at 23:44:45 UTC, Paul Backus wrote:

 Aren't macros in Lisp just regular Lisp code, too? :)

 As far as I'm concerned, a procedural macro is a function that

 1. runs at compile time,
 2. takes AST nodes as input, and
 3. produces AST nodes as output.

 Type functions fit those criteria perfectly. Currently, they 
 only work on a subset of AST nodes (types), so they're not a 
 *complete* implementation of procedural macros, but generalize 
 them enough and that's what you'll get.


 They don't exactly take AST nodes.
 They take a constrained (stabilized) datastructure which 
 reflects a subset of the ast.
 At least conceptually.
 They don't create ast nodes, but regular values, which may be 
 converted to ast-nodes in the case where they are statically 
 available. (currently only types)


This is also how macros work in some Lisp implementations. For 
example, in Racket, the AST nodes (called "syntax objects") are 
converted to S-expressions by the function `syntax->datum`, the 
S-expressions are manipulated by CTFE, and the result is 
converted back into an AST node by `datum->syntax`. [1]


 My currently exploration leads me to a path where you can 
 _extend_ open scopes within typefunction by using special 
 functions which map provide a stable mapping to compiler 
 intrinsics.
 such as __struct* __add_struct(string name, __symbol[] members);
 the __struct* will not be a proper struct type unless
 __type__ __add_to_scope(__scope Scope, __struct* s);
 has been called.


I haven't fully thought this through, but would it be possible to 
overload `mixin` to do this? E.g.,

     import core.ast: Struct, Type;

     // some type function
     Struct pair(Type t) { ... }

     alias PairOfInts = mixin(pair(int));

Conceptually, `mixin` already means "take this stuff and inject 
it into the current scope," and grammatically it's allowed to 
appear as both an expression and a type.


 At which point any modification of the __struct* is an error 
 and will abort compilation.


If __struct is only the "user-space" representation, and not the 
real AST node, I don't see why this is necessary, since modifying 
the __struct after mixing it in wouldn't have any affect on the 
actual AST.


 I am still searching for a solution which is more declarative, 
 but I fear for heavy usage (10_000 new types and declarations 
 added) scenarios the procedural imperative style is just more 
 optimizable.

 I am always open to syntax suggestions, as I don't like the one 
 I have :)


In Lisp, the declarative syntax is typically implemented as 
syntax sugar on top of the procedural syntax, using the 
`quasiquote` or `backquote` macro. [2][3] In D, one could imagine 
doing something similar with string mixins: generate the 
procedural code at compile time from a declarative DSL, then mix 
it in.

[1] https://docs.racket-lang.org/guide/stx-obj.html
[2] https://docs.racket-lang.org/guide/qq.html
[3] 
https://www.ccis.northeastern.edu/home/futrelle/teaching/lisp/cltl/clm/node190.html#BACKQUOTE
Feb 23 2021
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 23 February 2021 at 15:35:43 UTC, Paul Backus wrote:

 On Tuesday, 23 February 2021 at 14:14:13 UTC, Stefan Koch wrote:

 On Monday, 22 February 2021 at 23:44:45 UTC, Paul Backus wrote:


 I haven't fully thought this through, but would it be possible 
 to overload `mixin` to do this? E.g.,

     import core.ast: Struct, Type;

     // some type function
     Struct pair(Type t) { ... }

     alias PairOfInts = mixin(pair(int));

 Conceptually, `mixin` already means "take this stuff and inject 
 it into the current scope," and grammatically it's allowed to 
 appear as both an expression and a type.


Yes it would be possible to reuse mixin, but iirc mixins convert 
non-string expressions to string which would introduce ambiguity, 
if I special case it __struct, or __type__.


 At which point any modification of the __struct* is an error 
 and will abort compilation.


 If __struct is only the "user-space" representation, and not 
 the real AST node, I don't see why this is necessary, since 
 modifying the __struct after mixing it in wouldn't have any 
 affect on the actual AST.


The restriction is imposed to avoid confusing situations, due to 
the closeness of the mapping to real compiler entities you cannot 
finalize the same struct* twice and such things.
Modifying a __struct after it has been finalized or 'mixed-in' is 
either useless or a bug.
Feb 23 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 23 February 2021 at 15:49:14 UTC, Stefan Koch wrote:

 Yes it would be possible to reuse mixin, but iirc mixins 
 convert non-string expressions to string which would introduce 
 ambiguity, if I special case it __struct, or __type__.


My gut feeling is that it would not be too difficult to 
understand in practice, but yes, it would require special-casing.


 The restriction is imposed to avoid confusing situations, due 
 to the closeness of the mapping to real compiler entities you 
 cannot finalize the same struct* twice and such things.
 Modifying a __struct after it has been finalized or 'mixed-in' 
 is either useless or a bug.


Actually, now that I think of it, wouldn't the existing CTFE 
semantics prevent you from doing this anyway?

enum __struct s = someTypeFunction();
auto modify(ref __struct s) { ... }
enum result = modify(s); // Error: can't modify rvalue

Or if you tried to do it inside a type function:

auto someTypeFunction(...) {
     __struct s = ...;
     alias Before = mixin(s); // Error: can't read `s` at compile 
time
     modify(s);
}
Feb 23 2021
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 23 February 2021 at 16:10:26 UTC, Paul Backus wrote:

 On Tuesday, 23 February 2021 at 15:49:14 UTC, Stefan Koch wrote:

 [...]


 My gut feeling is that it would not be too difficult to 
 understand in practice, but yes, it would require 
 special-casing.


 [...]


 Actually, now that I think of it, wouldn't the existing CTFE 
 semantics prevent you from doing this anyway?

 enum __struct s = someTypeFunction();
 auto modify(ref __struct s) { ... }
 enum result = modify(s); // Error: can't modify rvalue

 Or if you tried to do it inside a type function:

 auto someTypeFunction(...) {
     __struct s = ...;
     alias Before = mixin(s); // Error: can't read `s` at 
 compile time
     modify(s);
 }


... yes you have a point there.

... perhaps I am worrying too much.
Feb 23 2021
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 22 February 2021 at 16:27:49 UTC, Andrei Alexandrescu 
wrote:

 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf


Hmm at a glance it looks close where I am at with typefunctions.

The syntactic difference is that the unary ^ is an either an 
implicit conversion or an explicit cast.

If they play their cards right and integrate their meta-info 
properly with constexpr it's going to beat D.
Feb 23 2021
parent reply IGotD- <nise nise.com> writes:
On Tuesday, 23 February 2021 at 13:58:34 UTC, Stefan Koch wrote:

 If they play their cards right and integrate their meta-info 
 properly with constexpr it's going to beat D.


Can you explain further why it is going to beat D (I assume in 
terms of functionality, not aesthetics).
Feb 23 2021
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 23 February 2021 at 15:04:46 UTC, IGotD- wrote:

 On Tuesday, 23 February 2021 at 13:58:34 UTC, Stefan Koch wrote:

 If they play their cards right and integrate their meta-info 
 properly with constexpr it's going to beat D.


 Can you explain further why it is going to beat D (I assume in 
 terms of functionality, not aesthetics).


Because the conversion to reflection values meta::info or 
whatever it was called happens inside the compiler.
It's not shackled to templates.

And therefore they can have very fast generation of these without 
inuring recursive evluate-semantic() function calls.
Feb 23 2021
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 23 February 2021 at 15:13:18 UTC, Stefan Koch wrote:

 On Tuesday, 23 February 2021 at 15:04:46 UTC, IGotD- wrote:

 On Tuesday, 23 February 2021 at 13:58:34 UTC, Stefan Koch 
 wrote:

 If they play their cards right and integrate their meta-info 
 properly with constexpr it's going to beat D.


 Can you explain further why it is going to beat D (I assume in 
 terms of functionality, not aesthetics).


 Because the conversion to reflection values meta::info or 
 whatever it was called happens inside the compiler.
 It's not shackled to templates.

 And therefore they can have very fast generation of these 
 without inuring recursive evluate-semantic() function calls.


Actually I take that back.
I should have looked more closely.
They have polymorphic expressions (changing the static type) 
which means they can only use those within polymorphic contexts 
... which does shackle them to templates again.
Feb 23 2021
prev sibling parent bitwise <bitwise.pvt gmail.com> writes:
On Monday, 22 February 2021 at 16:27:49 UTC, Andrei Alexandrescu 
wrote:

 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf




 void f(... [:range_of_types:] ...args)


This seems like it could provide a more intuitive intellisense 
display for things like std::make_shared, but without proper 
parameter names. Unless this provides a path to including 
parameter names, I don't see how it's better than <Args&&...>.
Feb 23 2021