digitalmars.dip.development - First Draft: Symbol Representation (Export)
- Richard (Rikki) Andrew Cattermole (37/37) Feb 29 D's support for shared libraries has historically presented
- Tim (17/29) Mar 08 Can a module, which is compiled as part of the application, still
- Richard (Rikki) Andrew Cattermole (11/43) Mar 08 Yes.
- Johan (9/21) Mar 29 Quick remark:
- Richard (Rikki) Andrew Cattermole (10/35) Mar 29 At the current point in time, I don't see any reason to introduce that
- Johan (9/21) Mar 29 "The compiler must not inline a function from an out of binary
- Richard (Rikki) Andrew Cattermole (10/36) Mar 29 That statement isn't just for pragma inline, it also applies to
- Mike Parker (7/17) Apr 10 This proposal is now officially DIP 1045. I've worked with Rikki
D's support for shared libraries has historically presented challenges for users, especially when dealing with complex use cases. This proposal aims to improve the user experience by establishing a user story that will provide a consistent and user-friendly approach to usage and compilation of binaries without a thorough understanding of linkers. With the recent resolution of dmd's exportation support on Windows (druntime support is still under development), we are now ready to focus on making D's shared library capabilities more accessible and usable for common use cases. Links: [current](https://gist.githubusercontent.com/rikkimax/1d7cfdb8ed74e9e4efc9ba0208f42e7e/11e36e8a332324c532379 da2835337d0306a6c9) [latest](https://gist.github.com/rikkimax/1d7cfdb8ed74e9e4efc9ba0208f42e7e) Some features of this proposal: - The ``export`` attribute is no longer a visibility modifier - A new switch is added ``-extI``, the external import path switch - Templates are not exported by default (prevents template emission bugs like [23255](https://issues.dlang.org/show_bug.cgi?id=23255) by requiring all templates in modules out of binary to instantiate) - The D interface generator will be able to produce .di files that are suitable for usage with both static libraries and shared libraries. - Gives a name to using export versus the visibility override switch as positive and negative notation - Officialize the ``Have_`` prefix used in dub and introduce ``InBinary_`` and ``Compiling_`` to enable very fined grained control over symbol mode as part of ``export`` - Generated symbols will be exported for a given encapsulation unit that has exportation There has already been quite a bit of back and forth between me and Walter with the aid of Adam Wilson (Thanks!), for this reason this proposal is skipping the idea step. In a previous version of this proposal a focus upon anything that affects shared libraries was taken. In this version of the proposal a focus upon things required for common use cases is taken, and when these are not taken direct failure to implement the use case in D occurs.
Feb 29
On Thursday, 29 February 2024 at 11:15:07 UTC, Richard (Rikki) Andrew Cattermole wrote:D's support for shared libraries has historically presented challenges for users, especially when dealing with complex use cases. This proposal aims to improve the user experience by establishing a user story that will provide a consistent and user-friendly approach to usage and compilation of binaries without a thorough understanding of linkers. With the recent resolution of dmd's exportation support on Windows (druntime support is still under development), we are now ready to focus on making D's shared library capabilities more accessible and usable for common use cases. Links: [current](https://gist.githubusercontent.com/rikkimax/1d7cfdb8ed74e9e4efc9ba0208f42e7e/11e36e8a332324c532379 da2835337d0306a6c9) [latest](https://gist.github.com/rikkimax/1d7cfdb8ed74e9e4efc9ba0208f42e7e)Can a module, which is compiled as part of the application, still contain prototypes for symbols from a DLL? Here is an example: ```D extern(C++) void foo(const char *s, size_t length); void foo(string s) { foo(s.ptr, s.length); } ``` Function `foo` is implemented in a C++ library, which could be compiled as a DLL. The module contains a prototype for the function, but also an overload with a more convenient interface for D. The module needs to be compiled, because it contains the overload. Can the prototype still be imported from a DLL and be part of the same module?
Mar 08
On 09/03/2024 5:49 AM, Tim wrote:On Thursday, 29 February 2024 at 11:15:07 UTC, Richard (Rikki) Andrew Cattermole wrote:Yes. Mangling and ABI is left untouched. About the only thing you need to deal with is setting a symbol into DllImport mode, which is already something you are meant to be doing. With this you can set it via ``export extern``, ``export(none)`` or when using an import path use ``-extI`` and have it ``export``. You can get away with using internal symbol mode because Windows will generate a wrapper function that is in internal mode as part of the import library. There is no difference in internal vs DllImport symbol modes on non-Windows.D's support for shared libraries has historically presented challenges for users, especially when dealing with complex use cases. This proposal aims to improve the user experience by establishing a user story that will provide a consistent and user-friendly approach to usage and compilation of binaries without a thorough understanding of linkers. With the recent resolution of dmd's exportation support on Windows (druntime support is still under development), we are now ready to focus on making D's shared library capabilities more accessible and usable for common use cases. Links: [current](https://gist.githubusercontent.com/rikkimax/1d7cfdb8ed74e9e4efc9ba0208f42e7e/11e36e8a332324c532379 da2835337d0306a6c9) [latest](https://gist.github.com/rikkimax/1d7cfdb8ed74e9e4efc9ba0208f42e7e)Can a module, which is compiled as part of the application, still contain prototypes for symbols from a DLL? Here is an example: ```D extern(C++) void foo(const char *s, size_t length); void foo(string s) { foo(s.ptr, s.length); } ``` Function `foo` is implemented in a C++ library, which could be compiled as a DLL. The module contains a prototype for the function, but also an overload with a more convenient interface for D. The module needs to be compiled, because it contains the overload. Can the prototype still be imported from a DLL and be part of the same module?
Mar 08
On Thursday, 29 February 2024 at 11:15:07 UTC, Richard (Rikki) Andrew Cattermole wrote:D's support for shared libraries has historically presented challenges for users, especially when dealing with complex use cases. This proposal aims to improve the user experience by establishing a user story that will provide a consistent and user-friendly approach to usage and compilation of binaries without a thorough understanding of linkers. With the recent resolution of dmd's exportation support on Windows (druntime support is still under development), we are now ready to focus on making D's shared library capabilities more accessible and usable for common use cases. Links: [current](https://gist.githubusercontent.com/rikkimax/1d7cfdb8ed74e9e4efc9ba0208f42e7e/11e36e8a332324c532379 da2835337d0306a6c9) [latest](https://gist.github.com/rikkimax/1d7cfdb8ed74e9e4efc9ba0208f42e7e)Quick remark: `export ( Identifier )` will have all the problems associated with version identifiers, in particular that you cannot combine version identifiers (A && B) or generate them briefly from boolean expressions. Is there a reason why you do not allow boolean expression here? -Johan
Mar 29
On 30/03/2024 2:28 AM, Johan wrote:On Thursday, 29 February 2024 at 11:15:07 UTC, Richard (Rikki) Andrew Cattermole wrote:At the current point in time, I don't see any reason to introduce that behavior. The version that would be provided should be coming from the compiler or your build manager. If you are not limiting yourself to ``InBinary_*``, you're probably going to have a lot of pain. It is meant for advanced situations where the external import path switch might be giving the wrong results and you need to go on a per symbol basis.D's support for shared libraries has historically presented challenges for users, especially when dealing with complex use cases. This proposal aims to improve the user experience by establishing a user story that will provide a consistent and user-friendly approach to usage and compilation of binaries without a thorough understanding of linkers. With the recent resolution of dmd's exportation support on Windows (druntime support is still under development), we are now ready to focus on making D's shared library capabilities more accessible and usable for common use cases. Links: [current](https://gist.githubusercontent.com/rikkimax/1d7cfdb8ed74e9e4efc9ba0208f42e7e/11e36e8a332324c532379 da2835337d0306a6c9) [latest](https://gist.github.com/rikkimax/1d7cfdb8ed74e9e4efc9ba0208f42e7e)Quick remark: `export ( Identifier )` will have all the problems associated with version identifiers, in particular that you cannot combine version identifiers (A && B) or generate them briefly from boolean expressions. Is there a reason why you do not allow boolean expression here? -Johan
Mar 29
On Thursday, 29 February 2024 at 11:15:07 UTC, Richard (Rikki) Andrew Cattermole wrote:D's support for shared libraries has historically presented challenges for users, especially when dealing with complex use cases. This proposal aims to improve the user experience by establishing a user story that will provide a consistent and user-friendly approach to usage and compilation of binaries without a thorough understanding of linkers. With the recent resolution of dmd's exportation support on Windows (druntime support is still under development), we are now ready to focus on making D's shared library capabilities more accessible and usable for common use cases. Links: [current](https://gist.githubusercontent.com/rikkimax/1d7cfdb8ed74e9e4efc9ba0208f42e7e/11e36e8a332324c532379 da2835337d0306a6c9) [latest](https://gist.github.com/rikkimax/1d7cfdb8ed74e9e4efc9ba0208f42e7e)"The compiler must not inline a function from an out of binary module if it refers to non-exported symbols." I think this should instead be detected at module compile time, giving a warning that an exported pragma(inline, true) function is calling a hidden function. Let's keep `pragma(inline, true)` as a very strong inlining hint. -Johan
Mar 29
On 30/03/2024 2:33 AM, Johan wrote:On Thursday, 29 February 2024 at 11:15:07 UTC, Richard (Rikki) Andrew Cattermole wrote:That statement isn't just for pragma inline, it also applies to preventing the backend from performing an inline too. However in saying that currently I have enough reasons to believe that honoring it will only result in linker errors that to users is just a garbled mess. Even with the warning from the compiler. I would need evidence that an inlined symbol accessing another binaries symbol that was not exported will work. Right now I have no evidence that this is possible. Therefore this change would allow non-linking code that had a zero percent chance to link, to pass compilation.D's support for shared libraries has historically presented challenges for users, especially when dealing with complex use cases. This proposal aims to improve the user experience by establishing a user story that will provide a consistent and user-friendly approach to usage and compilation of binaries without a thorough understanding of linkers. With the recent resolution of dmd's exportation support on Windows (druntime support is still under development), we are now ready to focus on making D's shared library capabilities more accessible and usable for common use cases. Links: [current](https://gist.githubusercontent.com/rikkimax/1d7cfdb8ed74e9e4efc9ba0208f42e7e/11e36e8a332324c532379 da2835337d0306a6c9) [latest](https://gist.github.com/rikkimax/1d7cfdb8ed74e9e4efc9ba0208f42e7e)"The compiler must not inline a function from an out of binary module if it refers to non-exported symbols." I think this should instead be detected at module compile time, giving a warning that an exported pragma(inline, true) function is calling a hidden function. Let's keep `pragma(inline, true)` as a very strong inlining hint. -Johan
Mar 29
On Thursday, 29 February 2024 at 11:15:07 UTC, Richard (Rikki) Andrew Cattermole wrote:D's support for shared libraries has historically presented challenges for users, especially when dealing with complex use cases. This proposal aims to improve the user experience by establishing a user story that will provide a consistent and user-friendly approach to usage and compilation of binaries without a thorough understanding of linkers. With the recent resolution of dmd's exportation support on Windows (druntime support is still under development), we are now ready to focus on making D's shared library capabilities more accessible and usable for common use cases.This proposal is now officially DIP 1045. I've worked with Rikki to revise it and it's with Walter and Atila for assessment. Unless they let me know they need more time, we should expect a decision before April 24th. https://github.com/dlang/DIPs/blob/master/DIPs/DIP1045.md
Apr 10