digitalmars.D - Dynamic libraries, again
- Jacob Carlborg (15/15) Mar 14 2010 I've been working the whole day on getting Tango to work built as a
- Walter Bright (6/22) Mar 14 2010 There will be an _minfo_beg and _minfo_end for each loaded shared
- Jacob Carlborg (4/27) Mar 15 2010 I've looked at the output dumpobj gives on Tango and there are no
- Jacob Carlborg (3/26) Mar 15 2010 Wouldn't this include all the modules and not just the modules that are
- Jacob Carlborg (13/36) Mar 15 2010 Latest update:
- Walter Bright (19/32) Mar 15 2010 Here's how things work. For each module, a reference to that module's Mo...
- Jacob Carlborg (14/55) Mar 15 2010 The beg and end sections are not in the dynamic library. But it's
- Walter Bright (18/85) Mar 15 2010 That doesn't make sense, because they are in the .o files
- Jacob Carlborg (9/100) Mar 15 2010 They are in the object files but not in the dynamic library. I verified
- Walter Bright (4/94) Mar 15 2010 I don't understand. What kind of linker errors?
- Jacob Carlborg (4/95) Mar 15 2010 I managed to get it to work now. I had to reverse the order of the
- Walter Bright (2/5) Mar 15 2010 Awesome! Thanks!
I've been working the whole day on getting Tango to work built as a dynamic library with dmd on Mac OS X. I have one problem (for now) that I can't seem to solve. The problem is that the module constructors are not called. That is because the module info array in the object module, which contains all the module constructors that should be called, is (almost) empty. It seems like that the compiler or the linker doesn't put the module constructors that are dynamically linked into the executable. I've also tried to move the _minfo_beg and _minfo_end variables, which are the beginning and the end of the sections containing the module info array, from the object module to a statically linked module and then passed them as arguments to the _moduleCtor function but without any success. Does anyone have any idea have to solve the above problem? Does the compiler need to be modified? /Jacob Carlborg
Mar 14 2010
Jacob Carlborg wrote:I've been working the whole day on getting Tango to work built as a dynamic library with dmd on Mac OS X. I have one problem (for now) that I can't seem to solve. The problem is that the module constructors are not called. That is because the module info array in the object module, which contains all the module constructors that should be called, is (almost) empty. It seems like that the compiler or the linker doesn't put the module constructors that are dynamically linked into the executable. I've also tried to move the _minfo_beg and _minfo_end variables, which are the beginning and the end of the sections containing the module info array, from the object module to a statically linked module and then passed them as arguments to the _moduleCtor function but without any success. Does anyone have any idea have to solve the above problem? Does the compiler need to be modified?There will be an _minfo_beg and _minfo_end for each loaded shared library. These must then be combined at runtime to form an aggregate array of all the modules. I don't know how to search the shared libraries to find the _minfo_beg/end sections, though I've never researched it, either.
Mar 14 2010
Jacob Carlborg <doob me.com> On 3/14/10 23:36, Walter Bright wrote:Jacob Carlborg wrote:I've looked at the output dumpobj gives on Tango and there are no __minfo_beg and __minfo_end sections but the __minfodata section is there.I've been working the whole day on getting Tango to work built as a dynamic library with dmd on Mac OS X. I have one problem (for now) that I can't seem to solve. The problem is that the module constructors are not called. That is because the module info array in the object module, which contains all the module constructors that should be called, is (almost) empty. It seems like that the compiler or the linker doesn't put the module constructors that are dynamically linked into the executable. I've also tried to move the _minfo_beg and _minfo_end variables, which are the beginning and the end of the sections containing the module info array, from the object module to a statically linked module and then passed them as arguments to the _moduleCtor function but without any success. Does anyone have any idea have to solve the above problem? Does the compiler need to be modified?There will be an _minfo_beg and _minfo_end for each loaded shared library. These must then be combined at runtime to form an aggregate array of all the modules.I don't know how to search the shared libraries to find the _minfo_beg/end sections, though I've never researched it, either.Haven't you written dumpobj? It does just that.
Mar 15 2010
On 3/14/10 23:36, Walter Bright wrote:Jacob Carlborg wrote:Wouldn't this include all the modules and not just the modules that are used?I've been working the whole day on getting Tango to work built as a dynamic library with dmd on Mac OS X. I have one problem (for now) that I can't seem to solve. The problem is that the module constructors are not called. That is because the module info array in the object module, which contains all the module constructors that should be called, is (almost) empty. It seems like that the compiler or the linker doesn't put the module constructors that are dynamically linked into the executable. I've also tried to move the _minfo_beg and _minfo_end variables, which are the beginning and the end of the sections containing the module info array, from the object module to a statically linked module and then passed them as arguments to the _moduleCtor function but without any success. Does anyone have any idea have to solve the above problem? Does the compiler need to be modified?There will be an _minfo_beg and _minfo_end for each loaded shared library. These must then be combined at runtime to form an aggregate array of all the modules.I don't know how to search the shared libraries to find the _minfo_beg/end sections, though I've never researched it, either.
Mar 15 2010
On 3/14/10 23:36, Walter Bright wrote:Jacob Carlborg wrote:Latest update: I've found the __minfodata section and the content looks fine. The new problem I have is that an exception is thrown in _moduleCtor2 and I can't catch the exception. The problem is that it can't find any handler table. I think it's something wrong with _deh_beg and _deh_end in the deh module. If I have the _deh_beg and _deh_end as globals (as they are by default) it doesn't find a handler table. If I instead get the __deh_beg and the __deh_end sections from the main executable it does find a table handler but the table looks very different compared to a statically linked Tango. The length of the table is close to 600 compared to 1 when Tango is statically linked. It also contains a lot of zeros.I've been working the whole day on getting Tango to work built as a dynamic library with dmd on Mac OS X. I have one problem (for now) that I can't seem to solve. The problem is that the module constructors are not called. That is because the module info array in the object module, which contains all the module constructors that should be called, is (almost) empty. It seems like that the compiler or the linker doesn't put the module constructors that are dynamically linked into the executable. I've also tried to move the _minfo_beg and _minfo_end variables, which are the beginning and the end of the sections containing the module info array, from the object module to a statically linked module and then passed them as arguments to the _moduleCtor function but without any success. Does anyone have any idea have to solve the above problem? Does the compiler need to be modified?There will be an _minfo_beg and _minfo_end for each loaded shared library. These must then be combined at runtime to form an aggregate array of all the modules. I don't know how to search the shared libraries to find the _minfo_beg/end sections, though I've never researched it, either.
Mar 15 2010
Jacob Carlborg wrote:Latest update: I've found the __minfodata section and the content looks fine. The new problem I have is that an exception is thrown in _moduleCtor2 and I can't catch the exception. The problem is that it can't find any handler table. I think it's something wrong with _deh_beg and _deh_end in the deh module. If I have the _deh_beg and _deh_end as globals (as they are by default) it doesn't find a handler table. If I instead get the __deh_beg and the __deh_end sections from the main executable it does find a table handler but the table looks very different compared to a statically linked Tango. The length of the table is close to 600 compared to 1 when Tango is statically linked. It also contains a lot of zeros.Here's how things work. For each module, a reference to that module's ModuleInfo instance is stored in a segment named __minfodata. But, also, two empty segments are created: __minfo_beg and __minfo_end. A symbol, _minfo_beg, is defined for section __minfo_beg, and _minfo_end for __minfo_end. A "sandwich" is created: segment __minfo_beg _minfo_beg is defined here segment __minfodata ModuleInfo pointers go here segment __minfo_end _minfo_end is defined here The linker helpfully groups all the __minfodata segments together, and voila, by examining &_minfo_beg and &_minfo_end, we know the beginning and end of the ModuleInfo array! The exception handling tables, __deh_beg, __deh_eh, __deh_end are handled exactly the same way. The code to generate these tables is in backend/machobj.c. So, for an application, this all works great. The trouble with a shared library is now there are two sets of these triples, and the runtime doesn't know about the shared library set. Yes, dumpobj knows about .o files, but it doesn't know about linked files.
Mar 15 2010
On 3/15/10 19:50, Walter Bright wrote:Jacob Carlborg wrote:The beg and end sections are not in the dynamic library. But it's possible to get the data with out those globals. This is how: http://tango.pastebin.com/YbmyZksx . This will also pull in all the module infos from the dynamic libraries regardless if they are used or not. BTW these globals are just causing trouble.Latest update: I've found the __minfodata section and the content looks fine. The new problem I have is that an exception is thrown in _moduleCtor2 and I can't catch the exception. The problem is that it can't find any handler table. I think it's something wrong with _deh_beg and _deh_end in the deh module. If I have the _deh_beg and _deh_end as globals (as they are by default) it doesn't find a handler table. If I instead get the __deh_beg and the __deh_end sections from the main executable it does find a table handler but the table looks very different compared to a statically linked Tango. The length of the table is close to 600 compared to 1 when Tango is statically linked. It also contains a lot of zeros.Here's how things work. For each module, a reference to that module's ModuleInfo instance is stored in a segment named __minfodata. But, also, two empty segments are created: __minfo_beg and __minfo_end. A symbol, _minfo_beg, is defined for section __minfo_beg, and _minfo_end for __minfo_end. A "sandwich" is created: segment __minfo_beg _minfo_beg is defined here segment __minfodata ModuleInfo pointers go here segment __minfo_end _minfo_end is defined hereThe linker helpfully groups all the __minfodata segments together, and voila, by examining &_minfo_beg and &_minfo_end, we know the beginning and end of the ModuleInfo array!I've already figured that out. I'm pretty sure the ModuleInfo array is working.The exception handling tables, __deh_beg, __deh_eh, __deh_end are handled exactly the same way. The code to generate these tables is in backend/machobj.c.This is the problem. I did the same with __deh_eh as with __minfodata (see the link above) but it doesn't work.So, for an application, this all works great. The trouble with a shared library is now there are two sets of these triples, and the runtime doesn't know about the shared library set.I think this is the problem with the __deh_eh section. I don't know from where I should get the __deh_eh section, the executable, the dynamic library or both. If both, how would I do that?Yes, dumpobj knows about .o files, but it doesn't know about linked files.dumpobj works fine with executables and shared libraries.
Mar 15 2010
Jacob Carlborg wrote:On 3/15/10 19:50, Walter Bright wrote:That doesn't make sense, because they are in the .o files (you can verify this with dumpobj). Did you generate a map file for the .so?Jacob Carlborg wrote:The beg and end sections are not in the dynamic library.Latest update: I've found the __minfodata section and the content looks fine. The new problem I have is that an exception is thrown in _moduleCtor2 and I can't catch the exception. The problem is that it can't find any handler table. I think it's something wrong with _deh_beg and _deh_end in the deh module. If I have the _deh_beg and _deh_end as globals (as they are by default) it doesn't find a handler table. If I instead get the __deh_beg and the __deh_end sections from the main executable it does find a table handler but the table looks very different compared to a statically linked Tango. The length of the table is close to 600 compared to 1 when Tango is statically linked. It also contains a lot of zeros.Here's how things work. For each module, a reference to that module's ModuleInfo instance is stored in a segment named __minfodata. But, also, two empty segments are created: __minfo_beg and __minfo_end. A symbol, _minfo_beg, is defined for section __minfo_beg, and _minfo_end for __minfo_end. A "sandwich" is created: segment __minfo_beg _minfo_beg is defined here segment __minfodata ModuleInfo pointers go here segment __minfo_end _minfo_end is defined hereBut it's possible to get the data with out those globals. This is how: http://tango.pastebin.com/YbmyZksx . This will also pull in all the module infos from the dynamic libraries regardless if they are used or not. BTW these globals are just causing trouble.How do they cause trouble, especially if they aren't even in the shared library?Does your code look in the deh for both the application and the shared library?The linker helpfully groups all the __minfodata segments together, and voila, by examining &_minfo_beg and &_minfo_end, we know the beginning and end of the ModuleInfo array!I've already figured that out. I'm pretty sure the ModuleInfo array is working.The exception handling tables, __deh_beg, __deh_eh, __deh_end are handled exactly the same way. The code to generate these tables is in backend/machobj.c.This is the problem. I did the same with __deh_eh as with __minfodata (see the link above) but it doesn't work.As I wrote, if you have a shared library, it will have its own deh section that is completely distinct from the application's. The reason is because the linker doesn't know what dynamic libraries will be loaded, so there's no way it can combine the deh sections. I don't think the runtime loader is capable of doing a link. Therefore, the exception handling code will have to search *both*. To do both, the shared library will have to 'register' its deh code with the application code that searches deh, so the application code will know there is another deh to search. Either that, or the application code will have to do a runtime search for all loaded deh sections.So, for an application, this all works great. The trouble with a shared library is now there are two sets of these triples, and the runtime doesn't know about the shared library set.I think this is the problem with the __deh_eh section. I don't know from where I should get the __deh_eh section, the executable, the dynamic library or both. If both, how would I do that?I've never tried that.Yes, dumpobj knows about .o files, but it doesn't know about linked files.dumpobj works fine with executables and shared libraries.
Mar 15 2010
On 3/15/10 20:36, Walter Bright wrote:Jacob Carlborg wrote:They are in the object files but not in the dynamic library. I verified that with dumpobj.On 3/15/10 19:50, Walter Bright wrote:That doesn't make sense, because they are in the .o files (you can verify this with dumpobj). Did you generate a map file for the .so?Jacob Carlborg wrote:The beg and end sections are not in the dynamic library.Latest update: I've found the __minfodata section and the content looks fine. The new problem I have is that an exception is thrown in _moduleCtor2 and I can't catch the exception. The problem is that it can't find any handler table. I think it's something wrong with _deh_beg and _deh_end in the deh module. If I have the _deh_beg and _deh_end as globals (as they are by default) it doesn't find a handler table. If I instead get the __deh_beg and the __deh_end sections from the main executable it does find a table handler but the table looks very different compared to a statically linked Tango. The length of the table is close to 600 compared to 1 when Tango is statically linked. It also contains a lot of zeros.Here's how things work. For each module, a reference to that module's ModuleInfo instance is stored in a segment named __minfodata. But, also, two empty segments are created: __minfo_beg and __minfo_end. A symbol, _minfo_beg, is defined for section __minfo_beg, and _minfo_end for __minfo_end. A "sandwich" is created: segment __minfo_beg _minfo_beg is defined here segment __minfodata ModuleInfo pointers go here segment __minfo_end _minfo_end is defined here1. They are globals 2. They cause linker errors when building dynamic libraries which requires workarounds 3. They're not put into the dynamic libraries as they shouldBut it's possible to get the data with out those globals. This is how: http://tango.pastebin.com/YbmyZksx . This will also pull in all the module infos from the dynamic libraries regardless if they are used or not. BTW these globals are just causing trouble.How do they cause trouble, especially if they aren't even in the shared library?Replying further downDoes your code look in the deh for both the application and the shared library?The linker helpfully groups all the __minfodata segments together, and voila, by examining &_minfo_beg and &_minfo_end, we know the beginning and end of the ModuleInfo array!I've already figured that out. I'm pretty sure the ModuleInfo array is working.The exception handling tables, __deh_beg, __deh_eh, __deh_end are handled exactly the same way. The code to generate these tables is in backend/machobj.c.This is the problem. I did the same with __deh_eh as with __minfodata (see the link above) but it doesn't work.I will try this now. Does the order matter?As I wrote, if you have a shared library, it will have its own deh section that is completely distinct from the application's. The reason is because the linker doesn't know what dynamic libraries will be loaded, so there's no way it can combine the deh sections. I don't think the runtime loader is capable of doing a link. Therefore, the exception handling code will have to search *both*. To do both, the shared library will have to 'register' its deh code with the application code that searches deh, so the application code will know there is another deh to search. Either that, or the application code will have to do a runtime search for all loaded deh sections.So, for an application, this all works great. The trouble with a shared library is now there are two sets of these triples, and the runtime doesn't know about the shared library set.I think this is the problem with the __deh_eh section. I don't know from where I should get the __deh_eh section, the executable, the dynamic library or both. If both, how would I do that?I've never tried that.Yes, dumpobj knows about .o files, but it doesn't know about linked files.dumpobj works fine with executables and shared libraries.
Mar 15 2010
Jacob Carlborg wrote:On 3/15/10 20:36, Walter Bright wrote:Perhaps that's related to the linker errors.Jacob Carlborg wrote:They are in the object files but not in the dynamic library. I verified that with dumpobj.On 3/15/10 19:50, Walter Bright wrote:That doesn't make sense, because they are in the .o files (you can verify this with dumpobj). Did you generate a map file for the .so?Jacob Carlborg wrote:The beg and end sections are not in the dynamic library.Latest update: I've found the __minfodata section and the content looks fine. The new problem I have is that an exception is thrown in _moduleCtor2 and I can't catch the exception. The problem is that it can't find any handler table. I think it's something wrong with _deh_beg and _deh_end in the deh module. If I have the _deh_beg and _deh_end as globals (as they are by default) it doesn't find a handler table. If I instead get the __deh_beg and the __deh_end sections from the main executable it does find a table handler but the table looks very different compared to a statically linked Tango. The length of the table is close to 600 compared to 1 when Tango is statically linked. It also contains a lot of zeros.Here's how things work. For each module, a reference to that module's ModuleInfo instance is stored in a segment named __minfodata. But, also, two empty segments are created: __minfo_beg and __minfo_end. A symbol, _minfo_beg, is defined for section __minfo_beg, and _minfo_end for __minfo_end. A "sandwich" is created: segment __minfo_beg _minfo_beg is defined here segment __minfodata ModuleInfo pointers go here segment __minfo_end _minfo_end is defined hereI don't understand. What kind of linker errors?1. They are globals 2. They cause linker errors when building dynamic libraries which requires workarounds 3. They're not put into the dynamic libraries as they shouldBut it's possible to get the data with out those globals. This is how: http://tango.pastebin.com/YbmyZksx . This will also pull in all the module infos from the dynamic libraries regardless if they are used or not. BTW these globals are just causing trouble.How do they cause trouble, especially if they aren't even in the shared library?No.I will try this now. Does the order matter?As I wrote, if you have a shared library, it will have its own deh section that is completely distinct from the application's. The reason is because the linker doesn't know what dynamic libraries will be loaded, so there's no way it can combine the deh sections. I don't think the runtime loader is capable of doing a link. Therefore, the exception handling code will have to search *both*. To do both, the shared library will have to 'register' its deh code with the application code that searches deh, so the application code will know there is another deh to search. Either that, or the application code will have to do a runtime search for all loaded deh sections.So, for an application, this all works great. The trouble with a shared library is now there are two sets of these triples, and the runtime doesn't know about the shared library set.I think this is the problem with the __deh_eh section. I don't know from where I should get the __deh_eh section, the executable, the dynamic library or both. If both, how would I do that?
Mar 15 2010
On 3/15/10 20:36, Walter Bright wrote:Jacob Carlborg wrote:I managed to get it to work now. I had to reverse the order of the module info array in object_.d and the function table array in deh.d. Thank you very much for all the help.On 3/15/10 19:50, Walter Bright wrote:That doesn't make sense, because they are in the .o files (you can verify this with dumpobj). Did you generate a map file for the .so?Jacob Carlborg wrote:The beg and end sections are not in the dynamic library.Latest update: I've found the __minfodata section and the content looks fine. The new problem I have is that an exception is thrown in _moduleCtor2 and I can't catch the exception. The problem is that it can't find any handler table. I think it's something wrong with _deh_beg and _deh_end in the deh module. If I have the _deh_beg and _deh_end as globals (as they are by default) it doesn't find a handler table. If I instead get the __deh_beg and the __deh_end sections from the main executable it does find a table handler but the table looks very different compared to a statically linked Tango. The length of the table is close to 600 compared to 1 when Tango is statically linked. It also contains a lot of zeros.Here's how things work. For each module, a reference to that module's ModuleInfo instance is stored in a segment named __minfodata. But, also, two empty segments are created: __minfo_beg and __minfo_end. A symbol, _minfo_beg, is defined for section __minfo_beg, and _minfo_end for __minfo_end. A "sandwich" is created: segment __minfo_beg _minfo_beg is defined here segment __minfodata ModuleInfo pointers go here segment __minfo_end _minfo_end is defined hereBut it's possible to get the data with out those globals. This is how: http://tango.pastebin.com/YbmyZksx . This will also pull in all the module infos from the dynamic libraries regardless if they are used or not. BTW these globals are just causing trouble.How do they cause trouble, especially if they aren't even in the shared library?Does your code look in the deh for both the application and the shared library?The linker helpfully groups all the __minfodata segments together, and voila, by examining &_minfo_beg and &_minfo_end, we know the beginning and end of the ModuleInfo array!I've already figured that out. I'm pretty sure the ModuleInfo array is working.The exception handling tables, __deh_beg, __deh_eh, __deh_end are handled exactly the same way. The code to generate these tables is in backend/machobj.c.This is the problem. I did the same with __deh_eh as with __minfodata (see the link above) but it doesn't work.As I wrote, if you have a shared library, it will have its own deh section that is completely distinct from the application's. The reason is because the linker doesn't know what dynamic libraries will be loaded, so there's no way it can combine the deh sections. I don't think the runtime loader is capable of doing a link. Therefore, the exception handling code will have to search *both*. To do both, the shared library will have to 'register' its deh code with the application code that searches deh, so the application code will know there is another deh to search. Either that, or the application code will have to do a runtime search for all loaded deh sections.So, for an application, this all works great. The trouble with a shared library is now there are two sets of these triples, and the runtime doesn't know about the shared library set.I think this is the problem with the __deh_eh section. I don't know from where I should get the __deh_eh section, the executable, the dynamic library or both. If both, how would I do that?I've never tried that.Yes, dumpobj knows about .o files, but it doesn't know about linked files.dumpobj works fine with executables and shared libraries.
Mar 15 2010
Jacob Carlborg wrote:I managed to get it to work now. I had to reverse the order of the module info array in object_.d and the function table array in deh.d. Thank you very much for all the help.Awesome! Thanks!
Mar 15 2010