digitalmars.D - C++/C mangleof inconsistency for OS X

Dan Olson (29/29) Apr 20 2015 An observation on OSX w/ 2.067: mangleof for C++ (and D) names produces

Jacob Carlborg (5/9) Apr 20 2015 I don't think it's intentional. The point of "mangleof" is to evaluate

Dan Olson (13/21) Apr 21 2015 Thanks Jacob.

Dan Olson (18/38) Apr 21 2015 Hmmm, I can see another point of view where mangleof should produce the

Jacob Carlborg (7/22) Apr 21 2015 Are there use cases where one would want to use some other mangling than...

Michel Fortin (11/24) Apr 21 2015 Apple does this in many of its own C headers. Lookup the definition of

Michel Fortin (13/61) Apr 21 2015 I think if you specify the mangling most of the time it's because you

Dan Olson (10/18) Apr 22 2015 I found that exists as core.demangle.mangleC() does exactly that, just

Michel Fortin (8/15) Apr 22 2015 And you think it's safe to assume all symbols on OS X will always have

Iain Buclaw via Digitalmars-d (8/19) Apr 22 2015 I think I've argued this before about pragma(mangle) - it's dangerous

Dan Olson (19/22) Apr 23 2015 So does LDC.

Iain Buclaw via Digitalmars-d (8/31) Apr 23 2015 There should *always* be a prefix there. ;-)

David Nadlinger (7/11) Apr 23 2015 For LLVM, this works by prefixing the name with \1.

Daniel Murphy (4/8) Apr 21 2015 Until fairly recently .mangleof was completely broken for eg extern(C++)...

Jacob Carlborg (5/9) Apr 22 2015 And what about pragma(mangle), should that output the symbol name

Daniel Murphy (2/4) Apr 22 2015 Yes.

Dan Olson (9/14) Apr 22 2015 Then for systems that add underscore prefix to compiler identifiers, dmd

Guillaume Chatelet (18/22) Apr 22 2015 This is still broken

Daniel Murphy (4/14) Apr 22 2015 That's not the issue we're talking about.

Iain Buclaw via Digitalmars-d (5/11) Apr 22 2015 Because of this, debugging in OSX is not as good as it could be (if

Dan Olson (8/11) Apr 23 2015 And I noticed __cxa_demangle() is defined that way. Target specific

Dan Olson <zans.is.for.cans yahoo.com> writes:

An observation on OSX w/ 2.067: mangleof for C++ (and D) names produces
the actual object file symbol while mangleof for C names strips a
leading underscore.

Is this intended?  If so what is rationale?

Demo

$ cat >showmangle.d <<XYZZY
void dfun(int);
extern(C++) void cxxfun(int);
extern (C) void cfun(int);

pragma(msg, dfun.mangleof);
pragma(msg, cxxfun.mangleof);
pragma(msg, cfun.mangleof);

void main()
{
    dfun(1);
    cxxfun(1);
    cfun(1);
}
XYZZY

$ dmd -c showmangle.d 
_D10showmangle4dfunFiZv
__Z6cxxfuni
cfun

$ nm showmangle.o | grep fun
                 U _D10showmangle4dfunFiZv
                 U __Z6cxxfuni
                 U _cfun

--
Dan

Apr 20 2015

Jacob Carlborg <doob me.com> writes:

On 2015-04-20 18:33, Dan Olson wrote:
 An observation on OSX w/ 2.067: mangleof for C++ (and D) names produces
 the actual object file symbol while mangleof for C names strips a
 leading underscore.

 Is this intended?  If so what is rationale?

I don't think it's intentional. The point of "mangleof" is to evaluate 
to the actual mangled name, as it appears in the object file.

-- 
/Jacob Carlborg

Apr 20 2015

Dan Olson <zans.is.for.cans yahoo.com> writes:

Jacob Carlborg <doob me.com> writes:

 On 2015-04-20 18:33, Dan Olson wrote:
 An observation on OSX w/ 2.067: mangleof for C++ (and D) names produces
 the actual object file symbol while mangleof for C names strips a
 leading underscore.

 Is this intended?  If so what is rationale?

 I don't think it's intentional. The point of "mangleof" is to evaluate
 to the actual mangled name, as it appears in the object file.

Thanks Jacob.

In that case, mangleof for extern(C) names on OS X and other systems
that add a leading underscore should include the underscore.

extern(C) int x;
version(linux) pragma(msg, foo.mangleof); // "x"
version(OSX) pragma(msg, foo.mangleof);   // "_x"

I'm trying to understand because ldc is different than dmd, and it is
related to proper debugging on systems with leading underscores.
pragma(mangle, name) is wrapped up in this too.  This needs to be right
to help D expand to other systems.
-- 
Dan

Apr 21 2015

Dan Olson <zans.is.for.cans yahoo.com> writes:

Dan Olson <zans.is.for.cans yahoo.com> writes:

 Jacob Carlborg <doob me.com> writes:

 On 2015-04-20 18:33, Dan Olson wrote:
 An observation on OSX w/ 2.067: mangleof for C++ (and D) names produces
 the actual object file symbol while mangleof for C names strips a
 leading underscore.

 Is this intended?  If so what is rationale?

 I don't think it's intentional. The point of "mangleof" is to evaluate
 to the actual mangled name, as it appears in the object file.

 Thanks Jacob.

 In that case, mangleof for extern(C) names on OS X and other systems
 that add a leading underscore should include the underscore.

 extern(C) int x;
 version(linux) pragma(msg, foo.mangleof); // "x"
 version(OSX) pragma(msg, foo.mangleof);   // "_x"

 I'm trying to understand because ldc is different than dmd, and it is
 related to proper debugging on systems with leading underscores.
 pragma(mangle, name) is wrapped up in this too.  This needs to be right
 to help D expand to other systems.

Hmmm, I can see another point of view where mangleof should produce the
equivalent extern(C) symbol.  My gut says this is the way it should
work.

If I want to call a C function void debug(const char*) from a C library,
I would do this because of D "debug" keyword:

  pragma(mangle, "debug")
  extern (C) void debug_c(const(char*));

Now I would think debug_c.mangleof -> "debug"
(and that is indeed what dmd produces even on OS X).

On systems which prepend an underscore, we want compiler to take care of
this so code is portable, otherwise code must do this:

version (OSX)
  pragma(mangle, "_debug") extern (C) void debug_c(const(char*));
else
  pragma(mangle, "debug") extern (C) void debug_c(const(char*));

--
Dan

Apr 21 2015

Jacob Carlborg <doob me.com> writes:

On 2015-04-21 19:01, Dan Olson wrote:

 Hmmm, I can see another point of view where mangleof should produce the
 equivalent extern(C) symbol.  My gut says this is the way it should
 work.

That makes sense.

 If I want to call a C function void debug(const char*) from a C library,
 I would do this because of D "debug" keyword:

    pragma(mangle, "debug")
    extern (C) void debug_c(const(char*));

 Now I would think debug_c.mangleof -> "debug"
 (and that is indeed what dmd produces even on OS X).

Are there use cases where one would want to use some other mangling than 
C? I mean, D is a system programing language.

 On systems which prepend an underscore, we want compiler to take care of
 this so code is portable, otherwise code must do this:

 version (OSX)
    pragma(mangle, "_debug") extern (C) void debug_c(const(char*));
 else
    pragma(mangle, "debug") extern (C) void debug_c(const(char*));

These are all good points.

-- 
/Jacob Carlborg

Apr 21 2015

Michel Fortin <michel.fortin michelf.ca> writes:

On 2015-04-21 18:29:36 +0000, Jacob Carlborg <doob me.com> said:

 On 2015-04-21 19:01, Dan Olson wrote:
 
 If I want to call a C function void debug(const char*) from a C library,
 I would do this because of D "debug" keyword:
 
    pragma(mangle, "debug")
    extern (C) void debug_c(const(char*));
 
 Now I would think debug_c.mangleof -> "debug"
 (and that is indeed what dmd produces even on OS X).

 
 Are there use cases where one would want to use some other mangling 
 than C? I mean, D is a system programing language.

Apple does this in many of its own C headers. Lookup the definition of 
pthread_join for instance, you'll see the __DARWIN_ALIAS macro which 
when expanded under certain circumstances adds a suffix to the symbol 
name in a similar way to pragma(mangle) in D. This allows some fixes to 
only apply to code compiled with newer SDKs. (Also note that the 
underscore is explicitly put there by the macro.)

-- 
Michel Fortin
michel.fortin michelf.ca
http://michelf.ca

Apr 21 2015

Michel Fortin <michel.fortin michelf.ca> writes:

On 2015-04-21 17:01:51 +0000, Dan Olson <zans.is.for.cans yahoo.com> said:

 Dan Olson <zans.is.for.cans yahoo.com> writes:
 
 Jacob Carlborg <doob me.com> writes:
 
 On 2015-04-20 18:33, Dan Olson wrote:
 An observation on OSX w/ 2.067: mangleof for C++ (and D) names produces
 the actual object file symbol while mangleof for C names strips a
 leading underscore.
 
 Is this intended?  If so what is rationale?

 
 I don't think it's intentional. The point of "mangleof" is to evaluate
 to the actual mangled name, as it appears in the object file.

 
 Thanks Jacob.
 
 In that case, mangleof for extern(C) names on OS X and other systems
 that add a leading underscore should include the underscore.
 
 extern(C) int x;
 version(linux) pragma(msg, foo.mangleof); // "x"
 version(OSX) pragma(msg, foo.mangleof);   // "_x"
 
 I'm trying to understand because ldc is different than dmd, and it is
 related to proper debugging on systems with leading underscores.
 pragma(mangle, name) is wrapped up in this too.  This needs to be right
 to help D expand to other systems.

 
 Hmmm, I can see another point of view where mangleof should produce the
 equivalent extern(C) symbol.  My gut says this is the way it should
 work.
 
 If I want to call a C function void debug(const char*) from a C library,
 I would do this because of D "debug" keyword:
 
   pragma(mangle, "debug")
   extern (C) void debug_c(const(char*));
 
 Now I would think debug_c.mangleof -> "debug"
 (and that is indeed what dmd produces even on OS X).
 
 On systems which prepend an underscore, we want compiler to take care of
 this so code is portable, otherwise code must do this:
 
 version (OSX)
   pragma(mangle, "_debug") extern (C) void debug_c(const(char*));
 else
   pragma(mangle, "debug") extern (C) void debug_c(const(char*));

I think if you specify the mangling most of the time it's because you 
don't want the compiler to do it for you. But you should consider doing 
this:

string mangleC(string name) {
	version (OSX) return "_" ~ name;
	else return name;
}

pragma(mangle, mangleC("debug")) extern (C) void debug_c(const(char*));

-- 
Michel Fortin
michel.fortin michelf.ca
http://michelf.ca

Apr 21 2015

Dan Olson <zans.is.for.cans yahoo.com> writes:

Michel Fortin <michel.fortin michelf.ca> writes:
 I think if you specify the mangling most of the time it's because you
 don't want the compiler to do it for you. But you should consider
 doing this:

 string mangleC(string name) {
 	version (OSX) return "_" ~ name;
 	else return name;
 }

 pragma(mangle, mangleC("debug")) extern (C) void debug_c(const(char*));

I found that exists as core.demangle.mangleC() does exactly that, just
added last fall.

Since the compile chain knows whether the target system prepends an
underscore, I wonder if it can be bubbled up into some compiler trait or
version that prevents writing versioned code based on system.  I think
it would help gdc much as it has many targets, and many prepend
underscore to symbols.  It would help with unittests like
compilable/cppmangle.d that would have to be tailored for every OS.
that uses gcc3 style C++ mangling.

Apr 22 2015

Michel Fortin <michel.fortin michelf.ca> writes:

On 2015-04-22 16:32:55 +0000, Dan Olson <zans.is.for.cans yahoo.com> said:

 Since the compile chain knows whether the target system prepends an
 underscore, I wonder if it can be bubbled up into some compiler trait or
 version that prevents writing versioned code based on system.  I think
 it would help gdc much as it has many targets, and many prepend
 underscore to symbols.  It would help with unittests like
 compilable/cppmangle.d that would have to be tailored for every OS.
 that uses gcc3 style C++ mangling.

And you think it's safe to assume all symbols on OS X will always have 
an underscore as a prefix just because symbols for C and C++ stuff do? 
(Hint: some Objective-C symbols don't start with "_".)

-- 
Michel Fortin
michel.fortin michelf.ca
http://michelf.ca

Apr 22 2015

Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:

On 22 April 2015 at 19:59, Michel Fortin via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 2015-04-22 16:32:55 +0000, Dan Olson <zans.is.for.cans yahoo.com> said:

 Since the compile chain knows whether the target system prepends an
 underscore, I wonder if it can be bubbled up into some compiler trait or
 version that prevents writing versioned code based on system.  I think
 it would help gdc much as it has many targets, and many prepend
 underscore to symbols.  It would help with unittests like
 compilable/cppmangle.d that would have to be tailored for every OS.
 that uses gcc3 style C++ mangling.


 And you think it's safe to assume all symbols on OS X will always have an
 underscore as a prefix just because symbols for C and C++ stuff do? (Hint:
 some Objective-C symbols don't start with "_".)

I think I've argued this before about pragma(mangle) - it's dangerous
to put any asserts in the testsuite surrounding this or to write any
library wrappers around it.

(FYI - last time I checked GDC goes ahead and applied target-specific
mangling on-top of pragma(mangle).  So you can relax and assume
nothing about the target).

Apr 22 2015

Dan Olson <zans.is.for.cans yahoo.com> writes:

Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
 (FYI - last time I checked GDC goes ahead and applied target-specific
 mangling on-top of pragma(mangle).  So you can relax and assume
 nothing about the target).

So does LDC.

Actually, DMD does too on OS X, but only for extern(C) - attention to
func plughc.

pragma(mangle, "plughc")   extern (C) void plughc();
pragma(mangle, "plughd")   extern(D) void plughd();
pragma(mangle, "plughcxx") extern(C++) void plughcxx();

pragma(msg, plughc.mangleof);
pragma(msg, plughd.mangleof);
pragma(msg, plughcxx.mangleof);

$ dmd -c showmangle.d
plughc
plughd
plughcxx

$ nm showmangle.o | grep plugh
                 U _plughc
                 U plughcxx
                 U plughd

Maybe this all needs to be left in implementation defined land?

Apr 23 2015

Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:

On 23 April 2015 at 19:15, Dan Olson via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
 (FYI - last time I checked GDC goes ahead and applied target-specific
 mangling on-top of pragma(mangle).  So you can relax and assume
 nothing about the target).

 So does LDC.

 Actually, DMD does too on OS X, but only for extern(C) - attention to
 func plughc.

 pragma(mangle, "plughc")   extern (C) void plughc();
 pragma(mangle, "plughd")   extern(D) void plughd();
 pragma(mangle, "plughcxx") extern(C++) void plughcxx();

 pragma(msg, plughc.mangleof);
 pragma(msg, plughd.mangleof);
 pragma(msg, plughcxx.mangleof);

 $ dmd -c showmangle.d
 plughc
 plughd
 plughcxx

 $ nm showmangle.o | grep plugh
                  U _plughc
                  U plughcxx
                  U plughd

 Maybe this all needs to be left in implementation defined land?

There should *always* be a prefix there. ;-)

A 'nice to know' which is specific to GDC.  If you prefix
pragma(mangle) names with "*" then it will not apply target-dependent
prefixes.

Regards
Iain

Apr 23 2015

"David Nadlinger" <code klickverbot.at> writes:

On Thursday, 23 April 2015 at 18:07:48 UTC, Iain Buclaw wrote:
 A 'nice to know' which is specific to GDC.  If you prefix
 pragma(mangle) names with "*" then it will not apply 
 target-dependent
 prefixes.

For LLVM, this works by prefixing the name with \1.

We need to do this internally for naked inline asm support on 
Windows to match DMD's mangling (stdcall-based cc, but no  <n> 
suffix).

This might not work from pragma(mangle) right now, though.

  — David

Apr 23 2015

"Daniel Murphy" <yebbliesnospam gmail.com> writes:

"Dan Olson"  wrote in message news:m28udmoj3v.fsf comcast.net...

 An observation on OSX w/ 2.067: mangleof for C++ (and D) names produces
 the actual object file symbol while mangleof for C names strips a
 leading underscore.

 Is this intended?  If so what is rationale?

Until fairly recently .mangleof was completely broken for eg extern(C++) 
mangling on windows.  I'm not surprised there are still bugs there.  Ideally 
.mangleof will give exactly the string that ends up in the object file.

Apr 21 2015

Jacob Carlborg <doob me.com> writes:

On 2015-04-22 03:46, Daniel Murphy wrote:

 Until fairly recently .mangleof was completely broken for eg extern(C++)
 mangling on windows.  I'm not surprised there are still bugs there.
 Ideally .mangleof will give exactly the string that ends up in the
 object file.

And what about pragma(mangle), should that output the symbol name 
completely untouched?

-- 
/Jacob Carlborg

Apr 22 2015

"Daniel Murphy" <yebbliesnospam gmail.com> writes:

"Jacob Carlborg"  wrote in message news:mh7mka$19r9$1 digitalmars.com... 

 And what about pragma(mangle), should that output the symbol name 
 completely untouched?

Yes.

Apr 22 2015

Dan Olson <zans.is.for.cans yahoo.com> writes:

"Daniel Murphy" <yebbliesnospam gmail.com> writes:

 "Jacob Carlborg"  wrote in message
 news:mh7mka$19r9$1 digitalmars.com... 

 And what about pragma(mangle), should that output the symbol name
 completely untouched?

 Yes.

Then for systems that add underscore prefix to compiler identifiers, dmd
.mangleof for extern(C) identifiers needs to include '_' prefix. I'll
try this locally on OS X and see what breaks.

It also means ldc merge-2.067, which just got C++ object symbols correct
with https://github.com/ldc-developers/ldc/pull/889, also needs to
include '_' prefix in .mangleof result for extern(C++).

There should be some helper to determine if compiler will add
underscores.  Otherwise I can see some unittests getting ugly.

Apr 22 2015

"Guillaume Chatelet" <chatelet.guillaume gmail.com> writes:

On Wednesday, 22 April 2015 at 01:46:57 UTC, Daniel Murphy wrote:
 Until fairly recently .mangleof was completely broken for eg 
 extern(C++) mangling on windows.  I'm not surprised there are 
 still bugs there.  Ideally .mangleof will give exactly the 
 string that ends up in the object file.

This is still broken
https://issues.dlang.org/show_bug.cgi?id=14178

I'm actually recoding the name mangling for linux/osx right now 
because the current implementation cannot work correctly with 
compression/substitutions (see 
http://mentorembedded.github.io/cxx-abi/abi.html#mangling-compression)

This is really tricky since substitution doesn't work the same 
when mangling template parameters or arguments. I'm making 
progress but there's a lot of gotchas. Especially if you start 
mixing namespace nesting, template instantiation and STL mangling 
substitutions.

You can have a look at the incomplete tests I have :
https://raw.githubusercontent.com/gchatelet/dmd/new_cpp_mangle/test/compilable/cppmangle.d

For instance this one is already pretty hard to understand :
_ZNK6nested21NestedTemplatedStructIiE23templatedMemberFunctionINS_12NestedStructEEEPT_PPKS4_
You can check the result with http://demangler.com/

Getting this right is a prerequisite to C++/STL integration.

Apr 22 2015

"Daniel Murphy" <yebbliesnospam gmail.com> writes:

"Guillaume Chatelet"  wrote in message 
news:vrcvkydsexiwamulangy forum.dlang.org...

 This is still broken
 https://issues.dlang.org/show_bug.cgi?id=14178

That's not the issue we're talking about.

 I'm actually recoding the name mangling for linux/osx right now because 
 the current implementation cannot work correctly with 
 compression/substitutions (see 
 http://mentorembedded.github.io/cxx-abi/abi.html#mangling-compression)

 This is really tricky since substitution doesn't work the same when 
 mangling template parameters or arguments. I'm making progress but there's 
 a lot of gotchas. Especially if you start mixing namespace nesting, 
 template instantiation and STL mangling substitutions.

Good!  Features have mostly just been patched in as people needed them.

Apr 22 2015

Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:

On 20 April 2015 at 18:33, Dan Olson via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 $ nm showmangle.o | grep fun
                  U _D10showmangle4dfunFiZv
                  U __Z6cxxfuni
                  U _cfun

 --
 Dan

Because of this, debugging in OSX is not as good as it could be (if
you use DMD).  All debuggers I've seen remove the leading '_' from
symbols before they begin demangling symbols.

Apr 22 2015

Dan Olson <zans.is.for.cans yahoo.com> writes:

Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
 Because of this, debugging in OSX is not as good as it could be (if
 you use DMD).  All debuggers I've seen remove the leading '_' from
 symbols before they begin demangling symbols.

And I noticed __cxa_demangle() is defined that way. Target specific
prefix must be removed for it to work.

I'd like to get debugging working better on OS X. I saw posts where you
improved GDB for D. I am going to try it out. Even though OS X has
switched to lldb, gdb is what I know, and what Emacs knows too. Working
on OS X debugging may help with understanding how D should deal with
target specific underscores.

Apr 23 2015

D Programming

C/C++ Programming

Other

digitalmars.D - C++/C mangleof inconsistency for OS X