www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Something like ADL from C++?

reply Manu <turkeyman gmail.com> writes:
Maybe someone has a pattern for doing this kind of thing...

So, I have a function that does something to a various set of things; let's
say we're going to serialise them or something:

```
module serialise;

void serialise(T)(void[] buffer, T)
  if (isSomeInt!T)
{ /* serialise integer */ }

void serialise(void[] buffer, const(char)[] str)
{ /* serialise string */ }

// ...etc
```

And some serialiser somewhere calls `serialise(thing)` for each thing,
where the type of thing chooses the right overload and we're all good...
sure, everyone knows this pattern.

So, I add a user thing in a module somewhere:

```
module app.user_thing:

struct UserThing
{ ... }

void serialise(void[] buffer, ref UserThing t)
{ /* serialise UserThing */ }
```

Now this thing wants to be serialisable, so you implement a serialise
function beside it...
In C++, this works; because ADL (argument dependent lookup) will cause to
additionally search the scope where the argument is defined for overloads.
Trouble is, in D unless `app.user_thing` was imported inside the serialiser
where it makes the call to `serialise()`, this overload won't be found,
because the UserThing overload is not in scope for the serialiser.

I tried to simulate something like ADL by getting `__traits(parent, value)`
in a loop until I find the module for non-builtin objects, and then have
the serialiser import that module prior to the call, to attempt to make
sure the argument's module is also in scope so any potential overloads can
be found when it tries to make the call:

import default_serialise : serialise;

void doSerialise(Things...)(void[] buffer, Things things)
{
  static foreach (thing; things)
  {{
    static if (isUserType!thing)
    {
      enum thingMod = getModuleForThing!thing;
      mixin(import " ~  thingMod ~ ";");
    }
    serialise(buffer thing);
  }}
}

The surprise is that if `thingMod` has a symbol `serialise`, it imports it
at the inner scope, and it shadows the global overload set rather than
complementing it...

So, I guess the thing I'm stuck on is; given there are some imports at
global scope, and it may have an overload set for some function; HOW can I
import more items to that overload set?

I tried this, but it doesn't work:

import default_serialise : serialise;

void doSerialise(Things...)(void[] buffer, Things things)
{
  static foreach (thing; things)
  {{
    static if (isUserType!thing)
    {
      enum thingMod = getModuleForThing!thing;
      mixin(import " ~  thingMod ~ ";");
      import default_serialise : serialise; // re-import at inner scope,
beside the other one
    }
    serialise(buffer thing);
  }}
}

Re-importing the globals into the same scope doesn't cause them to combine
either; just the symbol from whichever import statement appears first is
the winner...

Has anyone ever experimented with a pattern like this? Essentially, I can't
work out how to expand /combine an overload set to include symbols from
multiple imports....
Dec 03
next sibling parent reply Max Samukha <maxsamukha gmail.com> writes:
On Tuesday, 3 December 2024 at 11:55:50 UTC, Manu wrote:


 Re-importing the globals into the same scope doesn't cause them 
 to combine either; just the symbol from whichever import 
 statement appears first is the winner...

 Has anyone ever experimented with a pattern like this? 
 Essentially, I can't work out how to expand /combine an 
 overload set to include symbols from multiple imports....


You could merge overloads like this:

```
module a;

struct A
{
}

void serialise(void[], A)
{
     import std.stdio;
     writeln("A.serialize");
}

---
module b;

struct B
{
}

void serialise(void[], B)
{
     import std.stdio;
     writeln("B.serialise");
}

---
module default_serialise;

void serialise(T)(void[], T t)
{
     import std.stdio;
     writeln("T.serialise");
}

---
module main;

enum isUserType(T) = true; // change to whatever

// this is needed because we still cannot express "local to 
static foreach".
template serialiseOf(T)
{
     static if (is(__traits(parent, T) == module) && isUserType!T)
     {
         alias mod = __traits(parent, T);
         alias serialiseOf = mod.serialise;
     }
     else
     {
         static import default_serialise;
         alias serialiseOf = default_serialise.serialise;
     }
}

// this is needed because D doesn't allow overloading local 
functions.
template doSerialiseImpl(Things...)
{
     static foreach(Thing; Things)
         alias doSerialiseImpl = serialiseOf!Thing;
}

void doSerialise(Things...)(void[] buffer, Things things)
{
     static foreach (thing; things)
         doSerialiseImpl!Things(buffer, thing);
}

void main()
{
     import a, b;
     doSerialise(null, A(), B(), 42);
}
```

It never ceases to amaze me how difficult it still is to make 
such trivial things work.
Dec 03
next sibling parent Max Samukha <maxsamukha gmail.com> writes:
On Tuesday, 3 December 2024 at 15:17:47 UTC, Max Samukha wrote:



         static import default_serialise;
         alias serialiseOf = default_serialise.serialise;



The static import is a leftover from a previous iteration. You 
should not need it.
Dec 03
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/3/2024 7:17 AM, Max Samukha wrote:

 // this is needed because we still cannot express "local to static foreach".


That's because static foreach will become fairly useless if it introduced a 
scope. (Same as for static if.) However, you can still add { } to make a scope:

```
{ static foreach(...) { } }
```
Or:
```
static foreach (...) {{ }}
```
Dec 03
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12/3/24 23:16, Walter Bright wrote:

 On 12/3/2024 7:17 AM, Max Samukha wrote:

 // this is needed because we still cannot express "local to static 
 foreach".


 
 That's because static foreach will become fairly useless if it 
 introduced a scope. (Same as for static if.) However, you can still add 
 { } to make a scope:
 
 ```
 { static foreach(...) { } }
 ```
 Or:
 ```
 static foreach (...) {{ }}
 ```


It does introduce a scope for the loop variables, there is just no way 
to add your own temporary variables to that scope.

I had this implemented as the `__local` storage class, but I did not 
include it in the DIP proposal to avoid bikeshedding over syntax 
delaying the entire `static foreach` implementation:

https://github.com/dlang/dmd/commit/bac3426f9881e4a591df229423475efc6c3e0918

static foreach(i;0..10){
     __local enum tmp = i*i;
    // use tmp here
} // no redefinition error


There was also `__previous`, that allowed to access the `__local` 
variables from the previous iteration, allowing more imperative-style code:

https://github.com/dlang/dmd/commit/6c2f4b9c41cef5895efd3c1161920a47ed0b6464

static foreach(i;0..100){
     enum next_i = i+1;
     static if(i){
         static assert(__previous.i == i-1);
         static assert(__previous.next_i == i);
     }
}


It's a few lines of code to make that work though, the storage class 
exists, it's just not accessible to user code.

People have been using mixin templates to tuck away their 
loop-body-local variables ever since, due to the way scoping works for 
mixin templates (if you mix in the same mixin template multiple times 
unnamed into the same scope, the declarations in them do not conflict, 
but may become inaccessible from outside the mixin template instance.) 
This is what that comment was on.
Dec 03
next sibling parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 04/12/2024 3:32 PM, Timon Gehr wrote:

 I had this implemented as the |__local| storage class, but I did not 
 include it in the DIP proposal to avoid bikeshedding over syntax 
 delaying the entire |static foreach| implementation:


I'm trying to come up with alternative syntax.

Everything so far, a scope guard, an attribute, everything is coming up 
as having worse trade offs.

It may not feel right, but it has pretty good trade offs I think.

But yeah I'd quite like having this.
Dec 03
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
Thanks for the clarification
Dec 03
prev sibling next sibling parent Bradley Chatha <sealabjaster gmail.com> writes:
On Tuesday, 3 December 2024 at 11:55:50 UTC, Manu wrote:

 Has anyone ever experimented with a pattern like this? 
 Essentially, I can't work out how to expand /combine an 
 overload set to include symbols from multiple imports....


I gave this go using an eponymous template - it's definitely a 
bit yucky, but seems to do as you want (would obviously need more 
extensive work for real use cases rather than this simple POC):



```d
module serialise;

import std.traits : isNumeric;

string serialise(T)(T foo)
if(isNumeric!T)
{
     import std.conv : to;
     return foo.to!string();
}

string serialise(string s)
{
     return s;
}

string doSerialise(Things...)(Things things)
{
     string slowBuilder;

     static foreach (i, Thing; Things)
     {{
         enum hasSerialiser = __traits(compiles, 
serialiserFor!Thing);
         static if(hasSerialiser)
             slowBuilder ~= serialiserFor!Thing(things[i]);
         else
             slowBuilder ~= serialise(things[i]);
     }}

     return slowBuilder;
}

template serialiserFor(Thing)
{
     import std.traits : fullyQualifiedName;
     mixin("import thingMod = 
"~fullyQualifiedName!(__traits(parent, Thing))~";");
     alias serialiserFor = thingMod.serialise;
}
```



```d
module app;

import std.stdio;

void main()
{
	import serialise;
	import std : writeln;

	writeln(doSerialise(UserThing("a")));
	writeln(doSerialise("a"));
}

struct UserThing
{
	string a;
}

string serialise(UserThing thing)
{
	return "UserThing: " ~ thing.a;
}
```



```
UserThing: a
a
```

Namely:

The use of an eponymous template helps avoid the symbol 
resolution issue - you could also try to mixin a local import 
like `import userthing : userThingSerialise = serialise` instead, 
but a template might be cleaner.

I use a `__traits(compiles)` ~~abuse~~ check since it's simple, 
but you can probably also do some stuff with 
[__traits(hasMember)](https://dlang.org/spec/traits.html#hasMember),
`__traits(getMember)`, etc.

Another potential way if keeping the serialise function separate 
from the actual type is mandatory, which I can't be bothered to 
prototype, is to make a template struct similar to this:

```d
struct Serialiser(UserThingT, alias SerialiserFunc){...}
```

And then have a specific overload within the main serialiser 
module to handle this case:

```d
// Something like this at least.
yada serialise(Thing)(...)
if(isInstanceOf!(Serialiser, Thing))
{
   // Use SerialiserFunc against Thing?
}
```

Though there's probably a bunch of issues with lifetimes, const, 
ref, etc. with that approach.

Definitely an interesting issue, though personally I'd try my 
best to allow types to have a `serialise` function directly be a 
part of them rather than using free standing extentions.
Dec 03
prev sibling next sibling parent Max Samukha <maxsamukha gmail.com> writes:
On Tuesday, 3 December 2024 at 11:55:50 UTC, Manu wrote:


 import default_serialise : serialise;

 void doSerialise(Things...)(void[] buffer, Things things)
 {
   static foreach (thing; things)
   {{
     static if (isUserType!thing)
     {
       enum thingMod = getModuleForThing!thing;
       mixin(import " ~  thingMod ~ ";");
     }
     serialise(buffer thing);
   }}
 }


BTW, do you really need to create the overload set? You could 
call the right `serialize` directly if one exists in the type's 
module and fall back to the default one if it doesn't.
Dec 03
prev sibling next sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
This won't help you today but:

1. We are considering giving structs inheritance as part of replacing 
alias this.

2. I want a way to mark a method as 'reinterpreted' by the child 
class/struct. Specifically for serialization, removing the need for 
doing any lookups like this.
Dec 03
parent reply Derek Fawcus <dfawcus+dlang employees.org> writes:
On Tuesday, 3 December 2024 at 17:35:36 UTC, Richard (Rikki) 
Andrew Cattermole wrote:

 This won't help you today but:

 1. We are considering giving structs inheritance as part of 
 replacing alias this.

 2. I want a way to mark a method as 'reinterpreted' by the 
 child class/struct. Specifically for serialization, removing 
 the need for doing any lookups like this.


Please no.  Than then also makes the betterC subset too complex.

What facility of 'alias this' is it intended to preserve?  If 
only the implicit conversion, then a new form of operator, or 
special named member function strikes me as more suitable.

 From my perspective, one the the nice things about the class / 
strict difference is that struct does not have any classful 
behaviour, i.e. it has no inheritance.  If you add inheritance, 
that adds a thinking burden when reading code, and essentially 
removes the difference between class and struct, other than the 
implicit lock within classes.  I view the 'reference type' thing 
as a rather trivial difference.

One could add something like the struct embedding of 
Go/Limbo/Alef/Ken-C together with its implicit conversion for 
methods/functions.  The method is then interpreted by the 
embedded element, but against the embedded element, not the 
parent one.
Dec 03
parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 04/12/2024 7:22 AM, Derek Fawcus wrote:

 On Tuesday, 3 December 2024 at 17:35:36 UTC, Richard (Rikki) Andrew 
 Cattermole wrote:

 This won't help you today but:

 1. We are considering giving structs inheritance as part of replacing 
 alias this.

 2. I want a way to mark a method as 'reinterpreted' by the child 
 class/struct. Specifically for serialization, removing the need for 
 doing any lookups like this.


 
 Please no.  Than then also makes the betterC subset too complex.
 
 What facility of 'alias this' is it intended to preserve?  If only the 
 implicit conversion, then a new form of operator, or special named 
 member function strikes me as more suitable.
 
  From my perspective, one the the nice things about the class / strict 
 difference is that struct does not have any classful behaviour, i.e. it 
 has no inheritance.  If you add inheritance, that adds a thinking burden 
 when reading code, and essentially removes the difference between class 
 and struct, other than the implicit lock within classes.  I view the 
 'reference type' thing as a rather trivial difference.
 
 One could add something like the struct embedding of Go/Limbo/Alef/Ken-C 
 together with its implicit conversion for methods/functions.  The method 
 is then interpreted by the embedded element, but against the embedded 
 element, not the parent one.


My ideas post: 
https://forum.dlang.org/post/llqcjziyurwmyhzseonm forum.dlang.org

The problem is alias this is too complex, and has introduced bugs that 
cannot be fixed, as it is relied upon.

Struct inheritance alone cannot fix it (note I did not originally 
propose this, that was Walter), my proposal is to add a way to reparent 
the parent most type, to replicate a lot of use cases of alias this 
without the bad behavior.

As a follow user of -betterC, I want to get full D classes working in 
it. Because there is absolutely no reason why they should not work 
(although such things as Object root class would not be present, 
therefore would need to be explicit).

This may seem unnecessary, but removing the coupling between the 
language implementation and druntime is important for pay as you go 
users, and with that custom runtimes. Which offers us greater portability.
Dec 03
prev sibling next sibling parent reply Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Tuesday, 3 December 2024 at 11:55:50 UTC, Manu wrote:

 Has anyone ever experimented with a pattern like this? 
 Essentially, I can't work out how to expand /combine an 
 overload set to include symbols from multiple imports....


as a compromise you could have an uda that packs custom 
serialisation defined on type itself or field of that type, then 
you can make serializer employ it when it sees it somewhere.

Best regards,
Alexandru.
Dec 03
parent reply Derek Fawcus <dfawcus+dlang employees.org> writes:
On Tuesday, 3 December 2024 at 19:34:21 UTC, Alexandru Ermicioi 
wrote:

 On Tuesday, 3 December 2024 at 11:55:50 UTC, Manu wrote:

 Has anyone ever experimented with a pattern like this? 
 Essentially, I can't work out how to expand /combine an 
 overload set to include symbols from multiple imports....


 as a compromise you could have an uda that packs custom 
 serialisation defined on type itself or field of that type, 
 then you can make serializer employ it when it sees it 
 somewhere.


That is essentially what Go does for its serialisation (Marshal 
and Unmarshal) routines.

Only the public fields of a struct can be processed, and how to 
handle a field in a non default fashion is marked by an attribute 
- an extra string.

```Go
type DelSite struct {
     SiteID types.SiteID `json:"site-id" validate:"min=1"`
}
```

One of those is then encoded as follows, where 's' happens to be 
a struct ptr:

```Go
     var b []byte
     b, err := json.Marshal(s)
```

It is using run time reflection, but I see no obvious reason why 
compile time reflection could not be used instead, since that is 
what D has.
Dec 05
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 06/12/2024 8:07 AM, Derek Fawcus wrote:

 It is using run time reflection, but I see no obvious reason why compile 
 time reflection could not be used instead, since that is what D has.


I said this over in ideas, but the problem isn't controlling 
serialization, but the entry point into it.

Ideal:

```d
class Parent {
	void serialize() {
		// An implementation based upon typeof(this) goes here,
		//  currently this could only be Parent
	}
}

class Child : Parent {
}
```

How it has to be implemented today to work:


```d
class Parent {
	abstract void serialize();

	protected void serialize_(T:Parent)(T self) {
		
	}
}

class Child : Parent {
	void serialize() {
		super.serialize_(this);
	}
}
```

If you are forcing users of a parent class to add the call explicitly 
into the parent, its a major pain that other languages simply do not 
have. Not to mention easily forgotten.

This is what my attribute that I said I'm thinking about 
`` reinterptAsChild`` would solve.
Dec 05
parent reply Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Thursday, 5 December 2024 at 21:58:14 UTC, Richard (Rikki) 
Andrew Cattermole wrote:

 If you are forcing users of a parent class to add the call 
 explicitly into the parent, its a major pain that other 
 languages simply do not have. Not to mention easily forgotten.

 This is what my attribute that I said I'm thinking about 
 `` reinterptAsChild`` would solve.


I'm confused for example. What is the problem you're mentioning 
here?

If it is inheritance, then you either predefine the inheritance 
structure through annotations just like Jakson lib in java 
https://www.javadoc.io/doc/com.fasterxml.jackson.core/jackson-annotations/latest/com/fasterxml/jackson/annotatio
/JsonTypeInfo.html, or use runtime reflection to check what child it is and use
respective serializer for it that is prerecorded.
Dec 05
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 06/12/2024 1:47 PM, Alexandru Ermicioi wrote:

 On Thursday, 5 December 2024 at 21:58:14 UTC, Richard (Rikki) Andrew 
 Cattermole wrote:

 If you are forcing users of a parent class to add the call explicitly 
 into the parent, its a major pain that other languages simply do not 
 have. Not to mention easily forgotten.

 This is what my attribute that I said I'm thinking about 
 `` reinterptAsChild`` would solve.


 
 I'm confused for example. What is the problem you're mentioning here?
 
 If it is inheritance, then you either predefine the inheritance 
 structure through annotations just like Jakson lib in java https:// 
 www.javadoc.io/doc/com.fasterxml.jackson.core/jackson-annotations/ 
 latest/com/fasterxml/jackson/annotation/JsonTypeInfo.html, or use 
 runtime reflection to check what child it is and use respective 
 serializer for it that is prerecorded.


We do not have runtime reflection. We cannot copy languages that do, 
including for your example that isn't an alternative to runtime reflection.

```d
interface ISerialize {
	void serialize();
}

class SerializableRoot {
	void serialize() {
		// serialize here
	}
}

class MyType : SerializableRoot {
	 thing int field;

	...
}
```

Some how SerializableRoot.serialize has to be able to see the child 
MyType class. We have no good way of handling that currently.

The attribute allows the serialize method to think that its in MyType 
instead.
Dec 05
next sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Friday, 6 December 2024 at 01:18:03 UTC, Richard (Rikki) 
Andrew Cattermole wrote:

 Some how SerializableRoot.serialize has to be able to see the 
 child MyType class. We have no good way of handling that 
 currently.


There is the curious recurring template pattern of course.
Dec 05
prev sibling parent reply Arafel <er.krali gmail.com> writes:
On 6/12/24 2:18, Richard (Rikki) Andrew Cattermole wrote:

 We do not have runtime reflection. We cannot copy languages that do, 
 including for your example that isn't an alternative to runtime reflection.
 
 ```d
 interface ISerialize {
      void serialize();
 }
 
 class SerializableRoot {
      void serialize() {
          // serialize here
      }
 }
 
 class MyType : SerializableRoot {
       thing int field;
 
      ...
 }
 ```
 
 Some how SerializableRoot.serialize has to be able to see the child 
 MyType class. We have no good way of handling that currently.


```d
import std;

interface ISerialize {
     void serialize();
}

struct thing{ }

class SerializableRoot {
     void serialize(this T)() {
         // serialize here, possibly using __traits to introspect on T
         writeln("Serializing ", typeid(T));
     }
}

class MyType : SerializableRoot {
      thing int field;
}

void main() {
     MyType c = new MyType;
     c.serialize;
}
```
Dec 05
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 06/12/2024 8:59 PM, Arafel wrote:

 On 6/12/24 2:18, Richard (Rikki) Andrew Cattermole wrote:

 We do not have runtime reflection. We cannot copy languages that do, 
 including for your example that isn't an alternative to runtime 
 reflection.

 ```d
 interface ISerialize {
      void serialize();
 }

 class SerializableRoot {
      void serialize() {
          // serialize here
      }
 }

 class MyType : SerializableRoot {
       thing int field;

      ...
 }
 ```

 Some how SerializableRoot.serialize has to be able to see the child 
 MyType class. We have no good way of handling that currently.


 
 ```d
 import std;
 
 interface ISerialize {
      void serialize();
 }
 
 struct thing{ }
 
 class SerializableRoot {
      void serialize(this T)() {
          // serialize here, possibly using __traits to introspect on T
          writeln("Serializing ", typeid(T));
      }
 }
 
 class MyType : SerializableRoot {
       thing int field;
 }
 
 void main() {
      MyType c = new MyType;
      c.serialize;
 }
 ```


I'm fully aware of template this parameters, I was trying to get them to 
work 10 years ago.

They have problems that prevent their usage.

1. They are templates, so not in vtable
2. They require the this pointer to be typed to the child (extremely 
easy for this to not be the case, see 1)
3. Because they are not virtual, you cannot up cast (see 2)
4. Cannot be over ridden for any manual behavior

All and all, they become very complex for common usage, quite fragile if 
you don't use them right, not the kind of thing you recommend to people 
who are use to runtime reflection.
Dec 06
parent reply Arafel <er.krali gmail.com> writes:
On 6/12/24 9:05, Richard (Rikki) Andrew Cattermole wrote:

 I'm fully aware of template this parameters, I was trying to get them to 
 work 10 years ago.
 
 They have problems that prevent their usage.
 
 1. They are templates, so not in vtable
 2. They require the this pointer to be typed to the child (extremely 
 easy for this to not be the case, see 1)
 3. Because they are not virtual, you cannot up cast (see 2)
 4. Cannot be over ridden for any manual behavior
 
 All and all, they become very complex for common usage, quite fragile if 
 you don't use them right, not the kind of thing you recommend to people 
 who are use to runtime reflection.


That's true, I found a workaround using constructors and registering 
classes there, but currently it only works with one level of inheritance 
(without user intervention, the whole purpose of it):

```d
import std;

class SerializationRoot {
     static void function()[TypeInfo_Class] serializationMethods;
     // We would need a `shared static` constructor that would be always 
called
     /* shared static */ this(this T)() {
         auto myType = typeid(T);
         if (myType !in serializationMethods) {
             writeln("Registering type ", myType);
             serializationMethods[myType] = 
(&T.serializeDefaultImpl!T).funcptr;
         }
     }
     void serialize() {
         auto myType = typeid(this);
         assert(myType in serializationMethods, "The class was not 
registered for default serialization.");
         void delegate() dlg;
         dlg.funcptr = serializationMethods[myType];
         dlg.ptr = cast (void *) this;
         dlg();
         stdout.flush;
     }
     void serializeDefaultImpl(T)() {
         T realThis = cast(T) this;
         writeln("Default serialization for ", typeid(realThis));
     }
}

class MyClass : SerializationRoot { }

class MyOtherClass : SerializationRoot {
     this(int) { }
     override void serialize() {
         writeln("This class does something special, but still uses the 
default mechanism.");
         super.serialize;
     }
}

class MySpecialClass : SerializationRoot {
     override void serialize() {
         writeln("This class disregards everything takes everything into 
its own hands.");
     }
}

class MyProblematicClass : MyClass { }

void main() {
     SerializationRoot myClass = new MyClass;
     SerializationRoot myOtherClass = new MyOtherClass(1);
     SerializationRoot mySpecialClass = new MySpecialClass();
     SerializationRoot myProblematicClass = new MyProblematicClass();
     myClass.serialize;
     myOtherClass.serialize;
     mySpecialClass.serialize;
     myProblematicClass.serialize; // BOOM!
}
```

But it should work as expect if we had templated-this (shared) static 
constructors. My reading is that they are actually not forbidden in the 
spec ("member functions" should include static ones too) [1], and there 
is already an open bug for that [2] (with some duplicates).

Unfortunately I don't think it'll happen anytime soon, and I don't have 
the skills to have a go at it myself.

[1]: https://dlang.org/spec/template.html#template_this_parameter
[2]: https://issues.dlang.org/show_bug.cgi?id=10488
Dec 06
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 06/12/2024 10:46 PM, Arafel wrote:

 On 6/12/24 9:05, Richard (Rikki) Andrew Cattermole wrote:

 I'm fully aware of template this parameters, I was trying to get them 
 to work 10 years ago.

 They have problems that prevent their usage.

 1. They are templates, so not in vtable
 2. They require the this pointer to be typed to the child (extremely 
 easy for this to not be the case, see 1)
 3. Because they are not virtual, you cannot up cast (see 2)
 4. Cannot be over ridden for any manual behavior

 All and all, they become very complex for common usage, quite fragile 
 if you don't use them right, not the kind of thing you recommend to 
 people who are use to runtime reflection.


 
 That's true, I found a workaround using constructors and registering 
 classes there, but currently it only works with one level of inheritance 
 (without user intervention, the whole purpose of it):


Yes, once you have a level in the hierarchy that adds a constructor, you 
basically lose the ability to call that constructor with the template 
this parameter.

Which means the user derived type has to be final.

A struct starts to look inviting with all these issues present in 
classes lol
Dec 06
parent reply Arafel <er.krali gmail.com> writes:
On 6/12/24 11:18, Richard (Rikki) Andrew Cattermole wrote:

 Yes, once you have a level in the hierarchy that adds a constructor, you 
 basically lose the ability to call that constructor with the template 
 this parameter.


Shared static constructors with a template this parameter would be the 
solution (just promoting them in case somebody feels like implementing 
them ;-) )
Dec 06
parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 06/12/2024 11:38 PM, Arafel wrote:

 On 6/12/24 11:18, Richard (Rikki) Andrew Cattermole wrote:

 Yes, once you have a level in the hierarchy that adds a constructor, 
 you basically lose the ability to call that constructor with the 
 template this parameter.


 
 Shared static constructors with a template this parameter would be the 
 solution (just promoting them in case somebody feels like implementing 
 them ;-) )


For deserialization sure.

For serialization they cannot be overridden.

In both cases you have to use a single global registration (which may 
not be a down side).

So another case of it works for somethings, but not others.
Dec 06
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Koenig lookup? I ran away, far away, from that, after implementing it for C++.

It's a nightmare.
Dec 03
next sibling parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 04/12/2024 11:12 AM, Walter Bright wrote:

 Koenig lookup? I ran away, far away, from that, after implementing it 
 for C++.
 
 It's a nightmare.


I got a bit of a chuckle out of that.

You've been very careful with symbol lookup, its why I was pretty 
careful when designing my proposal for replacing alias this.

Keep things nice and simple, by following the inheritance hierarchy only!
Dec 03
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12/3/24 23:12, Walter Bright wrote:

 Koenig lookup? I ran away, far away, from that, after implementing it 
 for C++.
 
 It's a nightmare.


Sure, but the question was, how do you do actually do the serialization 
via introspection.
Dec 03
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/3/2024 6:39 PM, Timon Gehr wrote:

 Sure, but the question was, how do you do actually do the serialization via 
 introspection.


As I haven't attempted that, I might suggest use of:

1. interfaces

2. output ranges (sinks)
Dec 03
parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 04/12/2024 8:33 PM, Walter Bright wrote:

 On 12/3/2024 6:39 PM, Timon Gehr wrote:

 Sure, but the question was, how do you do actually do the 
 serialization via introspection.


 
 As I haven't attempted that, I might suggest use of:
 
 1. interfaces
 
 2. output ranges (sinks)


Don't forget template this parameter!

(its horrible)
Dec 03
prev sibling next sibling parent Kagamin <spam here.lot> writes:
On Wednesday, 4 December 2024 at 02:39:12 UTC, Timon Gehr wrote:

 Sure, but the question was, how do you do actually do the 
 serialization via introspection.



[ISerializable](https://learn.microsoft.com/en-us/dotnet/api/system.runtime.serializ
tion.iserializable) interface. There's also aspect-oriented approach, but it's
deprecated in favor of DTO pattern.
Dec 04
prev sibling parent Kagamin <spam here.lot> writes:
```
void serialize(byte[] o, string s){}

struct MyDto
{
	struct SerializerAspect
	{
		static void serialize(byte[] o, MyDto s){}
	}
}

void serialize2(T)(byte[] o, T s) if(!is(T.SerializerAspect))
{
	serialize(o,s);
}

void serialize2(T)(byte[] o, T s) if(is(T.SerializerAspect))
{
	T.SerializerAspect.serialize(o,s);
}

void test()
{
	byte[] o;
	string a;
	serialize2(o,a);
	MyDto c;
	serialize2(o,c);
}
```
Dec 04
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Back before I ended work on C++, a number of C++ Illuminati told me privately 
that ADL was a mistake. Things may have changed since then, and I don't know 
about that.
Dec 03
next sibling parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 04/12/2024 8:29 PM, Walter Bright wrote:

 Back before I ended work on C++, a number of C++ Illuminati told me 
 privately that ADL was a mistake. Things may have changed since then, 
 and I don't know about that.


https://en.cppreference.com/w/cpp/language/adl

Defects:

CWG 33	C++98
CWG 90	C++98
CWG 239	C++98
CWG 997	C++98
CWG 1690	C++98 C++11
CWG 1691	C++11
CWG 1692	C++98
CWG 2857	C++98

That list sure gives confidence!
Dec 03
prev sibling parent Manu <turkeyman gmail.com> writes:
On Wed, 4 Dec 2024 at 17:31, Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 Back before I ended work on C++, a number of C++ Illuminati told me
 privately
 that ADL was a mistake. Things may have changed since then, and I don't
 know
 about that.


I've used it effectively for decades and never had any problems.
I'm not advocating ADL necessarily, I can see the complexity; but we need
an effective response though. D has made the patterns it supports somewhat
more attractive than they are in C++ (mainly thanks to UFCS), yet the local
imports shadowing overload sets issue is confusing and unintuitive, and
makes the situation fail completely.

For my money, SFINAE is the most egregious bit of the C++ spec. (though
we're not doing a whole lot better!)
Dec 04
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On Wed, 4 Dec 2024 at 08:16, Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 Koenig lookup? I ran away, far away, from that, after implementing it for
 C++.

 It's a nightmare.


So what do you suggest?
UFCS makes this pattern more attractive than ever, but our lookup rules
inhibit any practical application.
The problem is essentially that a more-local import or singular declaration
shadows an entire overload set from a less-local scope, rather than merging
with it.
Dec 04
prev sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, December 4, 2024 7:31:50 PM MST Manu via Digitalmars-d wrote:

 On Wed, 4 Dec 2024 at 08:16, Walter Bright via Digitalmars-d <

 digitalmars-d puremagic.com> wrote:

 Koenig lookup? I ran away, far away, from that, after implementing it for
 C++.

 It's a nightmare.


 So what do you suggest?
 UFCS makes this pattern more attractive than ever, but our lookup rules
 inhibit any practical application.
 The problem is essentially that a more-local import or singular declaration
 shadows an entire overload set from a less-local scope, rather than merging
 with it.


The way that I've seen serialization done in D is to use UDAs, and the
(de)serializer uses type introspection to look at the UDAs to see what to
do. It could handle certain types by default (e.g. the built in language
types) and then extrapolate what to do for a type that is made up of known
types but isn't marked up with UDAs - or it could require that any types be
marked up to be (de)serialized. Either way, when dealing with a type that
doesn't have a UDA, the variable declaration using that type could use a UDA
to provide the required functions - or it could provide an intermediate type
(be it one that has a serialize and deseralize function that handles the
variable's type or because it's a type that that variable can be converted
to and from that the (de)serializer knows how to handle). IIRC, the
seralization stuff I saw in Java years ago did something similar (though
obviously with runtime reflection). Though obviously, C++ isn't going to use
that approach unless they've added some form of UDAs in a recent version,
and I missed it.

Obviously, that's a very different approach than what you're talking about,
and it may or may not fit what you're ultimately looking to do, but it's
what I've seen done - e.g. Mir has a serialization solution along those
lines.

- Jonathan M Davis
Dec 05
parent Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Thursday, 5 December 2024 at 13:16:06 UTC, Jonathan M Davis 
wrote:

 IIRC, the seralization stuff I saw in Java years ago did 
 something similar (though obviously with runtime reflection). 
 Though obviously, C++ isn't going to use that approach unless 
 they've added some form of UDAs in a recent version, and I 
 missed it.


Java serialisation libs still do employ annotations(UDA) for 
customising serialised output, and imho is much, much better than 
C++ approach.
Dec 05
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12/3/24 12:55, Manu wrote:

 Maybe someone has a pattern for doing this kind of thing...


There are a couple solutions that look okay already, but basically, 
importing into any unordered scope works, e.g.

```d
module default_serialise;
ubyte[] serialise(T)(T arg)if(!is(T==struct)){ return [1]; }
```

```d
module user_code;
struct S{}
ubyte[] serialise(S){ return [2]; }
```

```d
import default_serialise;

void main(){
     import std;
     static struct Dummy{
         import default_serialise: serialise;
         import user_code: serialise;
     }
     import user_code;
     writeln(Dummy.serialise(1)); // [1]
     writeln(Dummy.serialise(S())); // [2]
}
```

You can also use a dummy template scope (template Dummy(), and then 
`Dummy!().serialise`).

I dislike function-local import semantics quite a bit as they do not 
follow the well-thought-out overloading rules that apply to other imports.
Dec 03
next sibling parent Manu <turkeyman gmail.com> writes:
On Wed, 4 Dec 2024 at 12:56, Timon Gehr via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 12/3/24 12:55, Manu wrote:

 Maybe someone has a pattern for doing this kind of thing...


 There are a couple solutions that look okay already, but basically,
 importing into any unordered scope works, e.g.

 ```d
 module default_serialise;
 ubyte[] serialise(T)(T arg)if(!is(T==struct)){ return [1]; }
 ```

 ```d
 module user_code;
 struct S{}
 ubyte[] serialise(S){ return [2]; }
 ```

 ```d
 import default_serialise;

 void main(){
      import std;
      static struct Dummy{
          import default_serialise: serialise;
          import user_code: serialise;
      }
      import user_code;
      writeln(Dummy.serialise(1)); // [1]
      writeln(Dummy.serialise(S())); // [2]
 }
 ```

 You can also use a dummy template scope (template Dummy(), and then
 `Dummy!().serialise`).

 I dislike function-local import semantics quite a bit as they do not
 follow the well-thought-out overloading rules that apply to other imports.


"Unordered scope"; it never occurred to me that was a concept! But I can
see that now...
What a horrible hack, but I guess that could work.
I'll see if I can do anything with this idea...
Dec 04
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/3/2024 6:51 PM, Timon Gehr wrote:

 I dislike function-local import semantics quite a bit as they do not follow
the 
 well-thought-out overloading rules that apply to other imports.


I don't know if anyone remembers, but there was a huge debate about this in the 
early days. I had implemented local import rules just like all the other rules, 
but everyone else deemed that unintuitive, and so we now have a difficult to 
explain mechanism that is intuitive.
Dec 11
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 12/12/24 07:07, Walter Bright wrote:

 On 12/3/2024 6:51 PM, Timon Gehr wrote:

 I dislike function-local import semantics quite a bit as they do not 
 follow the well-thought-out overloading rules that apply to other 
 imports.


 
 I don't know if anyone remembers, but there was a huge debate about this 
 in the early days. I had implemented local import rules just like all 
 the other rules, but everyone else deemed that unintuitive, and so we 
 now have a difficult to explain mechanism that is intuitive.
 


I don't think I was around for that, but to be fair, it is unlikely that 
we are on exactly the same page here. There are multiple ways to do this 
that avoid hijacking problems and you have vehemently argued against 
overloading at function scope in the past.
Dec 12