www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - ModuleInfo, factories, and unittesting

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Compulsive ModuleInfo generation seems to be a drag. I'm not very 
familiar with the particulars but my understanding is ModuleInfo is 
needed for (a) Object.factory and (b) finding and running unittests.

Walter and I think Object.factory was a mistake and we'd like to make it 
opt-in long term (either by means of a build flag or a class attribute).

Until then, a very nice step forward is to NOT generate ModuleInfo if a 
module introduces no class.

For unittests, I figure things like static introspection should make it 
easier to enumerate and run unittests without a need for ModuleInfo.

What other issues/opportunities do you see related to ModuleInfo?


Thanks,

Andrei
Dec 18 2016
next sibling parent bitwise <bitwise.pvt gmail.com> writes:
On Sunday, 18 December 2016 at 16:53:24 UTC, Andrei Alexandrescu 
wrote:
 Compulsive ModuleInfo generation seems to be a drag. I'm not 
 very familiar with the particulars but my understanding is 
 ModuleInfo is needed for (a) Object.factory and (b) finding and 
 running unittests.

 Walter and I think Object.factory was a mistake and we'd like 
 to make it opt-in long term (either by means of a build flag or 
 a class attribute).

 Until then, a very nice step forward is to NOT generate 
 ModuleInfo if a module introduces no class.

 For unittests, I figure things like static introspection should 
 make it easier to enumerate and run unittests without a need 
 for ModuleInfo.

 What other issues/opportunities do you see related to 
 ModuleInfo?


 Thanks,

 Andrei
If ModuleInfo were to be opt in, would that mean it could be expanded then, to include things like class/struct fields?
Dec 18 2016
prev sibling next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 19/12/2016 5:53 AM, Andrei Alexandrescu wrote:
 Compulsive ModuleInfo generation seems to be a drag. I'm not very
 familiar with the particulars but my understanding is ModuleInfo is
 needed for (a) Object.factory and (b) finding and running unittests.

 Walter and I think Object.factory was a mistake and we'd like to make it
 opt-in long term (either by means of a build flag or a class attribute).

 Until then, a very nice step forward is to NOT generate ModuleInfo if a
 module introduces no class.

 For unittests, I figure things like static introspection should make it
 easier to enumerate and run unittests without a need for ModuleInfo.

 What other issues/opportunities do you see related to ModuleInfo?


 Thanks,

 Andrei
I've been toying with the idea of completely overhauling TypeInfo. Here is the gist of what I want: - -typeinfo=low/low-min/normal/high/none None is pretty much -betterC Low would be unittests + module constructors Low-min would be like low but with only the fields that is needed Normal is what we have now more or less High of course includes all the goodies like class fields and methods reflection - Full class + struct + union symbol reflection, so fields uda's ext. My feeling is D really is two different languages. An application one and a system one. The difference isn't much. It really comes down to how hooked into druntime it is and so TypeInfo. From there we just need to remove all language features that require druntime or allow a way to you know, clear deallocation path.
Dec 18 2016
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Sunday, 18 December 2016 at 22:55:30 UTC, rikki cattermole 
wrote:
 - -typeinfo=low/low-min/normal/high/none
   None is pretty much -betterC
   Low would be unittests + module constructors
   Low-min would be like low but with only the fields that is 
 needed
   Normal is what we have now more or less
   High of course includes all the goodies like class fields and 
 methods reflection
 - Full class + struct + union symbol reflection, so fields 
 uda's ext.
IMO, this seems like too much. I think none/minimal/full would be easier, where minimal would be equivalent to what's currently available right now. My specific concern is compatibility between compiled objects and knowing what symbols to expect. Also, just keeping the question of which level to use simple. If one object file was compiled without unittests, for example, a project as a whole could pass without error, but actually be broken..couldn't it?
Dec 18 2016
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 19/12/2016 12:37 PM, bitwise wrote:
 On Sunday, 18 December 2016 at 22:55:30 UTC, rikki cattermole wrote:
 - -typeinfo=low/low-min/normal/high/none
   None is pretty much -betterC
   Low would be unittests + module constructors
   Low-min would be like low but with only the fields that is needed
   Normal is what we have now more or less
   High of course includes all the goodies like class fields and
 methods reflection
 - Full class + struct + union symbol reflection, so fields uda's ext.
IMO, this seems like too much. I think none/minimal/full would be easier, where minimal would be equivalent to what's currently available right now. My specific concern is compatibility between compiled objects and knowing what symbols to expect. Also, just keeping the question of which level to use simple. If one object file was compiled without unittests, for example, a project as a whole could pass without error, but actually be broken..couldn't it?
Yes, if you don't include unittests via none, then it won't be tested. The only one of my suggestions that actually breaks typeinfo definitions is that of low-min. This is for memory constrained situations like kernels or MCU's but when you still want features such as unittesting and module constructors. All others will happily interlink without error. Overall, when the question is simple, the answer is too. Here the question isn't simple and so my answer isn't either. You've got to take into consideration that bloat depends on use case e.g. normal may very well be considered bloated by game dev standards while full is just about right for web applications.
Dec 18 2016
prev sibling next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 18 December 2016 at 16:53:24 UTC, Andrei Alexandrescu 
wrote:
 Compulsive ModuleInfo generation seems to be a drag. I'm not 
 very familiar with the particulars but my understanding is 
 ModuleInfo is needed for (a) Object.factory and (b) finding and 
 running unittests.

 Walter and I think Object.factory was a mistake and we'd like 
 to make it opt-in long term (either by means of a build flag or 
 a class attribute).
Don't forget that typeinfos are classes too, which makes the class attribute approach less attractive.
 Until then, a very nice step forward is to NOT generate 
 ModuleInfo if a module introduces no class.
Module (static) ctors & dtors need MI.
 For unittests, I figure things like static introspection should 
 make it easier to enumerate and run unittests without a need 
 for ModuleInfo.
We have __traits(getUnitTests,...) but the way to do DIY unittests is foreach(m; ModuleInfo) foreach(test; __traits(getUnitTests,m) test();
 What other issues/opportunities do you see related to 
 ModuleInfo?
AFAIK a module info is basically struct ModuleInfo { alias F = void function(); ModuleInfo* parent; string name; // may be static char array F s_ctor; // Static constructor F s_dtor; F ctor; // Thread local constructor F dtor; F unittester; //may be an array for each block void function(Object)[string] factory_mapping; //Foward to argless ctor, not sure what mapping is actually used. } If we optionally remove Object.factory, and move the unittests elsewhere (where to?) we are left with the ctors and dtors. If we store these as SoA then we minimise space (as few modules define *tors), but lose the mapping from Module -> *tor, but these are defined to be run in an implementation defined order anyway so no big loss. This is only a problem when dealing with multiple libraries compiled separately and dynamic libraries, but if the dynamic libs store their own SoA'd *tors and we merge the static libs Arrays at link time** then we have nothing useful left in the module info. If we could get factory_mapping mergeable at link-time then we would get a pay for what you use ** I don't know if this is possible, I don't mean merge duplicate symbols, I mean liba.a defines 3 module ctors, so does libb.a -> link == one ctor symbol 6 entries. Similar to what is done for C's global constructors, they end up in one big list.
 Thanks,

 Andrei
Dec 18 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/16 6:48 PM, Nicholas Wilson wrote:
 On Sunday, 18 December 2016 at 16:53:24 UTC, Andrei Alexandrescu wrote:
 Compulsive ModuleInfo generation seems to be a drag. I'm not very
 familiar with the particulars but my understanding is ModuleInfo is
 needed for (a) Object.factory and (b) finding and running unittests.

 Walter and I think Object.factory was a mistake and we'd like to make
 it opt-in long term (either by means of a build flag or a class
 attribute).
Don't forget that typeinfos are classes too, which makes the class attribute approach less attractive.
 Until then, a very nice step forward is to NOT generate ModuleInfo if
 a module introduces no class.
Module (static) ctors & dtors need MI.
Cool cool cool. Thanks.
 For unittests, I figure things like static introspection should make
 it easier to enumerate and run unittests without a need for ModuleInfo.
We have __traits(getUnitTests,...) but the way to do DIY unittests is foreach(m; ModuleInfo) foreach(test; __traits(getUnitTests,m) test();
Noice. Wait, the top foreach iterates what?
 What other issues/opportunities do you see related to ModuleInfo?
AFAIK a module info is basically struct ModuleInfo { alias F = void function(); ModuleInfo* parent; string name; // may be static char array F s_ctor; // Static constructor F s_dtor; F ctor; // Thread local constructor F dtor; F unittester; //may be an array for each block void function(Object)[string] factory_mapping; //Foward to argless ctor, not sure what mapping is actually used. } If we optionally remove Object.factory, and move the unittests elsewhere (where to?) we are left with the ctors and dtors. If we store these as SoA
SoA = ? Andrei
Dec 18 2016
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Monday, 19 December 2016 at 00:00:36 UTC, Andrei Alexandrescu 
wrote:
 On 12/18/16 6:48 PM, Nicholas Wilson wrote:
 [...]
Cool cool cool. Thanks.
         [...]
Noice. Wait, the top foreach iterates what?
Its a compile time magic loop I think. Both of them.
 [...]
SoA = ?
Structure of array. Think of a transpose of an array of structs, or consult the internet.
 Andrei
Dec 18 2016
parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 19 December 2016 at 00:15:24 UTC, Nicholas Wilson 
wrote:
 On Monday, 19 December 2016 at 00:00:36 UTC, Andrei 
 Alexandrescu wrote:
 On 12/18/16 6:48 PM, Nicholas Wilson wrote:
 [...]
Cool cool cool. Thanks.
         [...]
Noice. Wait, the top foreach iterates what?
Its a compile time magic loop I think. Both of them.
Nope, no magic. Just ModuleInfo.opApply in object.d. Atila
Dec 19 2016
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 19 December 2016 at 17:04:43 UTC, Atila Neves wrote:
 Nope, no magic. Just ModuleInfo.opApply in object.d.
Well, it is kinda magical because combining all the ModuleInfo takes some tricks from the linker. This method works with separate compilation, whereas the compile time tricks don't... and that's important to remember in any rewriting situations. Looping over the modules, all of them, including from separately compiled libraries, is necessary for static ctors, dtors, tests, and other things. Any replacement needs to keep that in mind somehow. The good news is we might be able to push it off onto the C runtime, if the order is done right. I'm not sure though.
Dec 19 2016
parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 19 December 2016 at 17:11:38 UTC, Adam D. Ruppe wrote:
 On Monday, 19 December 2016 at 17:04:43 UTC, Atila Neves wrote:
 Nope, no magic. Just ModuleInfo.opApply in object.d.
Well, it is kinda magical because combining all the ModuleInfo takes some tricks from the linker. This method works with separate compilation, whereas the compile time tricks don't... and that's important to remember in any rewriting situations.
I'd forgotten about separate compilation. The issue there is the algorithm for naming unittest blocks gives different results depending on how the code is compiled, so __traits(getUnitTests) unfortunately currently only works with "compile all the things!". I'd forgotten I wanted to change the compiler code responsible for the naming. Atila
Dec 19 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/19/16 12:25 PM, Atila Neves wrote:
 On Monday, 19 December 2016 at 17:11:38 UTC, Adam D. Ruppe wrote:
 On Monday, 19 December 2016 at 17:04:43 UTC, Atila Neves wrote:
 Nope, no magic. Just ModuleInfo.opApply in object.d.
Well, it is kinda magical because combining all the ModuleInfo takes some tricks from the linker. This method works with separate compilation, whereas the compile time tricks don't... and that's important to remember in any rewriting situations.
I'd forgotten about separate compilation. The issue there is the algorithm for naming unittest blocks gives different results depending on how the code is compiled, so __traits(getUnitTests) unfortunately currently only works with "compile all the things!". I'd forgotten I wanted to change the compiler code responsible for the naming.
Interesting, could you please add a detailed issue to bugzilla? Thanks! -- Andrei
Dec 19 2016
parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 19 December 2016 at 18:29:13 UTC, Andrei Alexandrescu 
wrote:
 On 12/19/16 12:25 PM, Atila Neves wrote:
 On Monday, 19 December 2016 at 17:11:38 UTC, Adam D. Ruppe 
 wrote:
 [...]
I'd forgotten about separate compilation. The issue there is the algorithm for naming unittest blocks gives different results depending on how the code is compiled, so __traits(getUnitTests) unfortunately currently only works with "compile all the things!". I'd forgotten I wanted to change the compiler code responsible for the naming.
Interesting, could you please add a detailed issue to bugzilla? Thanks! -- Andrei
https://issues.dlang.org/show_bug.cgi?id=16995 Atila
Dec 20 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/20/2016 01:15 PM, Atila Neves wrote:
 On Monday, 19 December 2016 at 18:29:13 UTC, Andrei Alexandrescu wrote:
 On 12/19/16 12:25 PM, Atila Neves wrote:
 On Monday, 19 December 2016 at 17:11:38 UTC, Adam D. Ruppe wrote:
 [...]
I'd forgotten about separate compilation. The issue there is the algorithm for naming unittest blocks gives different results depending on how the code is compiled, so __traits(getUnitTests) unfortunately currently only works with "compile all the things!". I'd forgotten I wanted to change the compiler code responsible for the naming.
Interesting, could you please add a detailed issue to bugzilla? Thanks! -- Andrei
https://issues.dlang.org/show_bug.cgi?id=16995
Gracias! -- Andrei
Dec 20 2016
prev sibling parent Atila Neves <atila.neves gmail.com> writes:
On Monday, 19 December 2016 at 00:00:36 UTC, Andrei Alexandrescu 
wrote:
 On 12/18/16 6:48 PM, Nicholas Wilson wrote:
 On Sunday, 18 December 2016 at 16:53:24 UTC, Andrei 
 Alexandrescu wrote:
[...]
Don't forget that typeinfos are classes too, which makes the class attribute approach less attractive.
[...]
Module (static) ctors & dtors need MI.
Cool cool cool. Thanks.
[...]
We have __traits(getUnitTests,...) but the way to do DIY unittests is foreach(m; ModuleInfo) foreach(test; __traits(getUnitTests,m) test();
Noice. Wait, the top foreach iterates what?
All the ModuleInfos in the binary. This confused the hell out of me when I was starting to learn D, since unit-threaded was my first project. Atila
Dec 19 2016
prev sibling next sibling parent Mike <none none.com> writes:
On Sunday, 18 December 2016 at 16:53:24 UTC, Andrei Alexandrescu 
wrote:
 What other issues/opportunities do you see related to 
 ModuleInfo?
I think it would be better to take a more holistic look at the relationship between the compiler and druntime. It's my understanding that many of the contracts between the compiler and druntime were written at at time when D didn't have things like templates and rich compile-time features. I think there is an opportunity to refactor the ModuleInfo, TypeInfo, and probably many other features, moving them out of the compiler into druntime (See http://forum.dlang.org/post/eiwalbqlbkipdrmsrfoh forum.dlang.org for a precise explanation). Then, with druntime's source code containing templates, conditional compilation, static-if, and the like, when the user compiles their code it only generates code that is actually being used. It could even be taken further with something like this (http://forum.dlang.org/post/psssnzurlzeqeneagora forum.dlang.org) where druntime's implementation is distributed as .di header files for compile-time verification and advantages beyond your original stated goal. Mike
Dec 18 2016
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2016-12-18 17:53, Andrei Alexandrescu wrote:

 For unittests, I figure things like static introspection should make it
 easier to enumerate and run unittests without a need for ModuleInfo.
We currently have no good way to collect unittest between modules. It's easy to collect them for a single specific module, but not globally. -- /Jacob Carlborg
Dec 18 2016
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2016-12-18 17:53, Andrei Alexandrescu wrote:
 Compulsive ModuleInfo generation seems to be a drag. I'm not very
 familiar with the particulars but my understanding is ModuleInfo is
 needed for (a) Object.factory and (b) finding and running unittests.

 Walter and I think Object.factory was a mistake and we'd like to make it
 opt-in long term (either by means of a build flag or a class attribute).

 Until then, a very nice step forward is to NOT generate ModuleInfo if a
 module introduces no class.

 For unittests, I figure things like static introspection should make it
 easier to enumerate and run unittests without a need for ModuleInfo.

 What other issues/opportunities do you see related to ModuleInfo?
Looking in the compiler, I see references to: * Coverage * Asserts * Arrays * Objective-C class info. If this is not generated Objective-C class will not work I'm not entirely sure what the above do. -- /Jacob Carlborg
Dec 18 2016
prev sibling next sibling parent Kagamin <spam here.lot> writes:
On Sunday, 18 December 2016 at 16:53:24 UTC, Andrei Alexandrescu 
wrote:
 What other issues/opportunities do you see related to 
 ModuleInfo?
IIRC TypeInfo is generated for C++ classes.
Dec 19 2016
prev sibling next sibling parent Atila Neves <atila.neves gmail.com> writes:
On Sunday, 18 December 2016 at 16:53:24 UTC, Andrei Alexandrescu 
wrote:
 Compulsive ModuleInfo generation seems to be a drag. I'm not 
 very familiar with the particulars but my understanding is 
 ModuleInfo is needed for (a) Object.factory and (b) finding and 
 running unittests.

 Walter and I think Object.factory was a mistake and we'd like 
 to make it opt-in long term (either by means of a build flag or 
 a class attribute).

 Until then, a very nice step forward is to NOT generate 
 ModuleInfo if a module introduces no class.

 For unittests, I figure things like static introspection should 
 make it easier to enumerate and run unittests without a need 
 for ModuleInfo.
That is correct, and how unit-threaded works. Atila
Dec 19 2016
prev sibling next sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Sunday, 18 December 2016 at 16:53:24 UTC, Andrei Alexandrescu 
wrote:
 Compulsive ModuleInfo generation seems to be a drag. I'm not 
 very familiar with the particulars but my understanding is 
 ModuleInfo is needed for (a) Object.factory and (b) finding and 
 running unittests.

 Walter and I think Object.factory was a mistake and we'd like 
 to make it opt-in long term (either by means of a build flag or 
 a class attribute).

 Until then, a very nice step forward is to NOT generate 
 ModuleInfo if a module introduces no class.

 For unittests, I figure things like static introspection should 
 make it easier to enumerate and run unittests without a need 
 for ModuleInfo.

 What other issues/opportunities do you see related to 
 ModuleInfo?


 Thanks,

 Andrei
Factory without auto generated info is possible. It needs works. Tried before: https://github.com/dlang/phobos/pull/4062
Dec 19 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/19/16 1:25 PM, Basile B. wrote:
 On Sunday, 18 December 2016 at 16:53:24 UTC, Andrei Alexandrescu wrote:
 Compulsive ModuleInfo generation seems to be a drag. I'm not very
 familiar with the particulars but my understanding is ModuleInfo is
 needed for (a) Object.factory and (b) finding and running unittests.

 Walter and I think Object.factory was a mistake and we'd like to make
 it opt-in long term (either by means of a build flag or a class
 attribute).

 Until then, a very nice step forward is to NOT generate ModuleInfo if
 a module introduces no class.

 For unittests, I figure things like static introspection should make
 it easier to enumerate and run unittests without a need for ModuleInfo.

 What other issues/opportunities do you see related to ModuleInfo?


 Thanks,

 Andrei
Factory without auto generated info is possible. It needs works. Tried before: https://github.com/dlang/phobos/pull/4062
Thanks! -- Andrei
Dec 19 2016
prev sibling parent reply Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D
Subject: Re: ModuleInfo, factories, and unittesting
References: <o36etv$1dkf$1 digitalmars.com>
In-Reply-To: <o36etv$1dkf$1 digitalmars.com>

--7wvBWNf1eEV3Om5J2XAUlXpdicHPOUn18
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 12/18/2016 06:53 PM, Andrei Alexandrescu wrote:
 Compulsive ModuleInfo generation seems to be a drag. I'm not very
 familiar with the particulars but my understanding is ModuleInfo is
 needed for (a) Object.factory and (b) finding and running unittests.
=20
 Walter and I think Object.factory was a mistake and we'd like to make i=
t
 opt-in long term (either by means of a build flag or a class attribute)=
=2E
=20
 Until then, a very nice step forward is to NOT generate ModuleInfo if a=
 module introduces no class.
=20
 For unittests, I figure things like static introspection should make it=
 easier to enumerate and run unittests without a need for ModuleInfo.
=20
 What other issues/opportunities do you see related to ModuleInfo?
=20
=20
 Thanks,
=20
 Andrei
We rely on ModuleInfo for custom test runner in ocean (https://github.com/sociomantic-tsunami/ocean/blob/v2.x.x/src/ocean/core/= UnitTestRunner.d). Static introspection can't provide same functionality in general case (even if all bugs will be fixed) because it requires either maintaining list of all modules or ensuring everything is imported from some root module. Former is inconvenience, latter is simply not true for libraries (though can be worked around by build system). --7wvBWNf1eEV3Om5J2XAUlXpdicHPOUn18--
Dec 20 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/20/2016 12:01 AM, Dicebot wrote:
 We rely on ModuleInfo for custom test runner in ocean
 (https://github.com/sociomantic-tsunami/ocean/blob/v2.x.x/src/ocean/core/UnitTestRunner.d).
 Static introspection can't provide same functionality in general case
 (even if all bugs will be fixed) because it requires either maintaining
 list of all modules or ensuring everything is imported from some root
 module. Former is inconvenience, latter is simply not true for libraries
 (though can be worked around by build system).
Does that simply need to find all the unit tests? (There are ways to do that without ModuleInfo, it's just that it's convenient and portable with ModuleInfo.)
Dec 20 2016
parent reply Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D
Subject: Re: ModuleInfo, factories, and unittesting
References: <o36etv$1dkf$1 digitalmars.com> <o3aogl$2imd$1 digitalmars.com>
 <o3bthc$2har$1 digitalmars.com>
In-Reply-To: <o3bthc$2har$1 digitalmars.com>

--54x3M2MJMGsO0KsWBao2A3c0Jw6162Hlh
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 12/20/2016 08:33 PM, Walter Bright wrote:
 On 12/20/2016 12:01 AM, Dicebot wrote:
 We rely on ModuleInfo for custom test runner in ocean
 (https://github.com/sociomantic-tsunami/ocean/blob/v2.x.x/src/ocean/co=
re/UnitTestRunner.d).
 Static introspection can't provide same functionality in general case
 (even if all bugs will be fixed) because it requires either maintainin=
g
 list of all modules or ensuring everything is imported from some root
 module. Former is inconvenience, latter is simply not true for librari=
es
 (though can be worked around by build system).
=20 Does that simply need to find all the unit tests? (There are ways to do=
 that without ModuleInfo, it's just that it's convenient and portable
 with ModuleInfo.)
Yes, pretty much. What ways do you have in mind? I am only aware of two: 1) ModuleInfo 2) https://dlang.org/spec/traits.html#getUnitTests --54x3M2MJMGsO0KsWBao2A3c0Jw6162Hlh--
Dec 20 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/20/2016 11:05 AM, Dicebot wrote:
 Yes, pretty much. What ways do you have in mind? I am only aware of two:

 1) ModuleInfo
 2) https://dlang.org/spec/traits.html#getUnitTests
Put pointers to them in a special segment.
Dec 20 2016
next sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Tue, 20 Dec 2016 12:36:53 -0800
schrieb Walter Bright <newshound2 digitalmars.com>:

 On 12/20/2016 11:05 AM, Dicebot wrote:
 Yes, pretty much. What ways do you have in mind? I am only aware of
 two:

 1) ModuleInfo
 2) https://dlang.org/spec/traits.html#getUnitTests  
Put pointers to them in a special segment.
You need some kind of linker support to do this to provide the start/end symbols. Binutils has got a nice feature to do that automatically for you though: https://sourceware.org/binutils/docs/ld/Orphan-Sections.html " If an orphaned section's name is representable as a C identifier then the linker will automatically see PROVIDE two symbols: __start_SECNAME and __stop_SECNAME, where SECNAME is the name of the section." However, things get a little more complicated when shared libraries are involved. You'll have to make the __start/__stop symbols private to the library (that's easy) and somehow provide a function in every library to access the library private __start/__stop symbols. That's basically how we assemble the list of moduleinfos in rt.sections. Back to topic: I'd really love to see a generalization of RTInfo/mixin templates in D: ---------------------------------------------------------------------- auto mixin template scanUnittests(Module) { static if (hasUnittest!Module) { static this() { foreach(test, getUnittests!Module) test(); } } } ---------------------------------------------------------------------- Whenever you import a module containing an auto mixin the compiler would mixin the declaration into the module. This should be incredibly powerful for serialization, std.benchmark and all kind of introspection tasks. (You then still need some way to pass this information to the 'runtime' world. Either static this, C-style ctors or custom data sections are possibilities. OTOH a mixin can also define members that are accessible from the outside if the module name is known)
Dec 21 2016
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2016-12-21 18:43, Johannes Pfau wrote:

 Back to topic: I'd really love to see a generalization of RTInfo/mixin
 templates in D:
I implemented RTInfo for modules a couple of years ago [1]. Unfortunately it was rejected because it had the same problem as RTInfo, it only works inside object.d. I had some ideas for that as well but it was not liked either. [1] https://github.com/dlang/dmd/pull/2271 -- /Jacob Carlborg
Dec 21 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/21/2016 9:43 AM, Johannes Pfau wrote:
 You need some kind of linker support to do this to provide the
 start/end symbols.
That's partially correct. I've done this for decades by having the compiler itself emit those symbols. There are other tricks one can do, such as putting the pointers into the exception handler tables sections.
Dec 21 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/21/2016 11:24 PM, Walter Bright wrote:
 On 12/21/2016 9:43 AM, Johannes Pfau wrote:
 You need some kind of linker support to do this to provide the
 start/end symbols.
That's partially correct. I've done this for decades by having the compiler itself emit those symbols. There are other tricks one can do, such as putting the pointers into the exception handler tables sections.
Or have the compiler call a "registerUnittest()" function with a parameter that's the pointer to the unittest info. Of course, that would require that a registerUnittest function exists somewhere.
Dec 22 2016
parent reply Piotrek <piotrek unknownuniverse.net> writes:
On Thursday, 22 December 2016 at 09:10:53 UTC, Walter Bright 
wrote:
 On 12/21/2016 11:24 PM, Walter Bright wrote:
 On 12/21/2016 9:43 AM, Johannes Pfau wrote:
 You need some kind of linker support to do this to provide the
 start/end symbols.
That's partially correct. I've done this for decades by having the compiler itself emit those symbols. There are other tricks one can do, such as putting the pointers into the exception handler tables sections.
Or have the compiler call a "registerUnittest()" function with a parameter that's the pointer to the unittest info. Of course, that would require that a registerUnittest function exists somewhere.
I don't know what other people think but the current status of inability to give test a name (plus selective unittesting) and continue on failure is puzzling to me. Cheers, Piotrek
Dec 22 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/22/2016 12:46 PM, Piotrek wrote:
 The inability to give test a name (plus selective unittesting) and
 continue on failure is puzzling to me.
Both of these are affordable with the current language (attributes) with changes to druntime. Who wants to take this? -- Andrei
Dec 22 2016
next sibling parent John Colvin <john.loughran.colvin gmail.com> writes:
On Thursday, 22 December 2016 at 18:19:31 UTC, Andrei 
Alexandrescu wrote:
 On 12/22/2016 12:46 PM, Piotrek wrote:
 The inability to give test a name (plus selective unittesting) 
 and
 continue on failure is puzzling to me.
Both of these are affordable with the current language (attributes) with changes to druntime. Who wants to take this? -- Andrei
I find Atila Neves' unit-threaded useful for running single unittests when using dub, perhaps some inspiration can be found there https://github.com/atilaneves/unit-threaded
Dec 22 2016
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2016-12-22 19:19, Andrei Alexandrescu wrote:

 Both of these are affordable with the current language (attributes) with
 changes to druntime. Who wants to take this? -- Andrei
What do you think about this idea [1]? [1] http://forum.dlang.org/post/npptbk$2mk0$1 digitalmars.com -- /Jacob Carlborg
Dec 22 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/23/2016 02:37 AM, Jacob Carlborg wrote:
 On 2016-12-22 19:19, Andrei Alexandrescu wrote:

 Both of these are affordable with the current language (attributes) with
 changes to druntime. Who wants to take this? -- Andrei
What do you think about this idea [1]? [1] http://forum.dlang.org/post/npptbk$2mk0$1 digitalmars.com
It looks like a solid traditional approach. It would be great to explore approaches where D's unique features would bring a strategic advantage. -- Andrei
Dec 23 2016
parent Jacob Carlborg <doob me.com> writes:
On 2016-12-23 18:12, Andrei Alexandrescu wrote:

 It looks like a solid traditional approach. It would be great to explore
 approaches where D's unique features would bring a strategic advantage.
I had hoped that D's unique features good be used on top of that low level library. -- /Jacob Carlborg
Dec 24 2016
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Thursday, 22 December 2016 at 18:19:31 UTC, Andrei 
Alexandrescu wrote:
 On 12/22/2016 12:46 PM, Piotrek wrote:
 The inability to give test a name (plus selective unittesting) 
 and
 continue on failure is puzzling to me.
Both of these are affordable with the current language (attributes) with changes to druntime. Who wants to take this? -- Andrei
I would, but I don't see how this is possible without one of: 1) A standardised build system 2) Changes to the language Attributes mean static reflection, and that means explicitly (and at compile-time) stating every module being built somewhere. There is no current way to reflect on packages. The only way I know how to do this is to use the build system to autogenerate a D file listing all modules to reflect on, which not coincidentally is how unit-threaded does it, with dub. But it only works if you're using dub and even then fails sometimes if it can't parse `dub describe` properly. I would've changed druntime 4 years ago if I could, it's the only reason I know of ModuleInfo and what it has to do with unit tests. Maybe I'm missing something or lacking imagination. Atila
Dec 23 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/23/2016 06:18 AM, Atila Neves wrote:
 On Thursday, 22 December 2016 at 18:19:31 UTC, Andrei Alexandrescu wrote:
 On 12/22/2016 12:46 PM, Piotrek wrote:
 The inability to give test a name (plus selective unittesting) and
 continue on failure is puzzling to me.
Both of these are affordable with the current language (attributes) with changes to druntime. Who wants to take this? -- Andrei
I would, but I don't see how this is possible without one of: 1) A standardised build system 2) Changes to the language Attributes mean static reflection, and that means explicitly (and at compile-time) stating every module being built somewhere. There is no current way to reflect on packages.
Oh, I see. So there needs to be some sort of cooperation between the compile-time realm (stash the unittest attributes during compilation) and the run-time realm (pick up that data and use it).
 The only way I know how to do this
 is to use the build system to autogenerate a D file listing all modules
 to reflect on, which not coincidentally is how unit-threaded does it,
 with dub. But it only works if you're using dub and even then fails
 sometimes if it can't parse `dub describe` properly.
Got it.
 I would've changed druntime 4 years ago if I could, it's the only reason
 I know of ModuleInfo and what it has to do with unit tests.

 Maybe I'm missing something or lacking imagination.
Well clearly some changes in the compiler and/or runtime could bring us to where we want to be. Andrei
Dec 23 2016
parent Jacob Carlborg <doob me.com> writes:
On 2016-12-23 18:09, Andrei Alexandrescu wrote:

 Well clearly some changes in the compiler and/or runtime could bring us
 to where we want to be.
Yeah, something like RTInfo but for modules. There's already an issue reported for this since a couple of years ago [1]. [1] https://issues.dlang.org/show_bug.cgi?id=10023 -- /Jacob Carlborg
Dec 24 2016
prev sibling next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Thursday, 22 December 2016 at 17:46:06 UTC, Piotrek wrote:
 On Thursday, 22 December 2016 at 09:10:53 UTC, Walter Bright 
 wrote:
 On 12/21/2016 11:24 PM, Walter Bright wrote:
[...]
Or have the compiler call a "registerUnittest()" function with a parameter that's the pointer to the unittest info. Of course, that would require that a registerUnittest function exists somewhere.
I don't know what other people think but the current status of inability to give test a name (plus selective unittesting) and continue on failure is puzzling to me. Cheers, Piotrek
The worst is how useless plain `assert` is. But, all of these issues can (and have) be solved by libraries. Atila
Dec 23 2016
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Friday, 23 December 2016 at 11:21:00 UTC, Atila Neves wrote:
 The worst is how useless plain `assert` is.
I would love it, LOVE it, if assert just printed its info. Consider the following code: int a = 0; int b = 1; assert(a == b); Wouldn't it just be glorious if that magically printed: test.d(4): Assertion failure (a == b) is false a = 0 b = 1 I'd also settle for it just showing the left hand and right hand side of the comparison. assert((0 && 1) == b); prints: `false == 1 failed` but it would be REALLY nice if it printed the expression then recursed down and found all the various variable things and printed them too. assert is built in for a reason, I thought, why then is our built in assert crappier than C's library assert?! Let's make it use the compiler's inside info. Let's make it respond accordingly to its environment (so like assert in contracts may be subtly different than in main bodies or unittests etc)
Dec 23 2016
next sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Fri, 23 Dec 2016 14:20:44 +0000, Adam D. Ruppe wrote:
 Wouldn't it just be glorious if that magically printed:
 
 test.d(4): Assertion failure
      (a == b) is false a = 0 b = 1
I *think* you can do that with an ugly API and string mixins and only needing to parse to the level of parentheses and equality operators.
Dec 23 2016
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Friday, 23 December 2016 at 14:43:28 UTC, Chris Wright wrote:
 I *think* you can do that with an ugly API and string mixins 
 and only needing to parse to the level of parentheses and 
 equality operators.
Yeah, you kinda can, or, of course, you can do stuff like assertEquals etc. named functions, or string template args like the existing libraries do. Just since it is a built in, we can do so much more! The main hard part would be getting the printing support in the runtime, but meh you can go SUCH a long way just printing the built ins, and it doesn't take that much code.... and it could be under conditional compilation to make it opt out too. sometimes i confess i want to write a new druntime. maybe i'd call it "dango"
Dec 23 2016
parent Jacob Carlborg <doob me.com> writes:
On 2016-12-23 16:09, Adam D. Ruppe wrote:

 sometimes i confess i want to write a new druntime. maybe i'd call it
 "dango"
Hehe, druntime is the only part that was shared between Tango and Phobos/druntime for D2. -- /Jacob Carlborg
Dec 24 2016
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2016-12-23 15:20, Adam D. Ruppe wrote:

 Wouldn't it just be glorious if that magically printed:

 test.d(4): Assertion failure
      (a == b) is false
      a = 0
      b = 1
Or if there was a language feature that allowed us to do this in a library, and a bunch of other things. I wonder what that could be ... :) -- /Jacob Carlborg
Dec 24 2016
parent Seb <seb wilzba.ch> writes:
On Saturday, 24 December 2016 at 09:31:42 UTC, Jacob Carlborg 
wrote:
 On 2016-12-23 15:20, Adam D. Ruppe wrote:

 Wouldn't it just be glorious if that magically printed:

 test.d(4): Assertion failure
      (a == b) is false
      a = 0
      b = 1
Or if there was a language feature that allowed us to do this in a library, and a bunch of other things. I wonder what that could be ... :)
As no one has mentioned it yet - some ideas are preserved in DIP83: http://wiki.dlang.org/DIP83
Dec 24 2016
prev sibling parent reply Piotrek <piotrek unknownuniverse.net> writes:
On Friday, 23 December 2016 at 11:21:00 UTC, Atila Neves wrote:
 The worst is how useless plain `assert` is. But, all of these 
 issues can (and have) be solved by libraries.

 Atila
Would assert fixing take into account it's presence in betterC code? Cheers, Piotrek
Dec 23 2016
parent reply Atila Neves <atila.neves gmail.com> writes:
On Friday, 23 December 2016 at 16:28:58 UTC, Piotrek wrote:
 On Friday, 23 December 2016 at 11:21:00 UTC, Atila Neves wrote:
 The worst is how useless plain `assert` is. But, all of these 
 issues can (and have) be solved by libraries.

 Atila
Cheers, Piotrek Would assert fixing take into account it's presence in betterC code?
Why would you care if the unittest build is betterC or not? The main build, sure, but it really shouldn't matter if the unit tests need druntime or the whole kitchen sink. Atila
Dec 23 2016
parent Piotrek <piotrek unknownuniverse.net> writes:
On Friday, 23 December 2016 at 17:07:29 UTC, Atila Neves wrote:
 On Friday, 23 December 2016 at 16:28:58 UTC, Piotrek wrote:
 On Friday, 23 December 2016 at 11:21:00 UTC, Atila Neves wrote:
 The worst is how useless plain `assert` is. But, all of these 
 issues can (and have) be solved by libraries.

 Atila
Cheers, Piotrek Would assert fixing take into account it's presence in betterC code?
Why would you care if the unittest build is betterC or not? The main build, sure, but it really shouldn't matter if the unit tests need druntime or the whole kitchen sink. Atila
I was referring to "assert" call in general here. I don't have enough experience for all betterC requirements. And I think I have never used unit tests and betterC switch together. Cheers, Piotrek
Dec 23 2016
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 22 December 2016 at 17:46:06 UTC, Piotrek wrote:
 I don't know what other people think but the current status of 

 inability to give test a name (plus selective unittesting) and  
 continue on failure is puzzling to me.
Have you seen my filthy hack for getting individual unittests to continue on failure? http://stackoverflow.com/a/40896271/1457000 It'd be a lot easier though to just actually get the compiler or library to do it. There's so many ways, I personally like the RTInfo for modules approach, it is easy to implement, easy to customize per project, and gives us access to the CT features. But there's others too.
Dec 23 2016
parent reply Piotrek <piotrek unknownuniverse.net> writes:
On Friday, 23 December 2016 at 14:06:24 UTC, Adam D. Ruppe wrote:
 Have you seen my filthy hack for getting individual unittests 
 to continue on failure? 
 http://stackoverflow.com/a/40896271/1457000
I have to say you are a master of D hacks :) This code can potentially reprogram a CPU and break my HW! pragma(joking, off).
 It'd be a lot easier though to just actually get the compiler 
 or library to do it. There's so many ways, I personally like 
 the RTInfo for modules approach, it is easy to implement, easy 
 to customize per project, and gives us access to the CT 
 features. But there's others too.
And that's why I said *quick development*. In theory, I can add any feature to any language by myself with a certain level of investment. But it's not the reality for 99,999% (all poor and lazy) programmers. In result I have to accept small obstacles and go on. Otherwise I wouldn't go anywhere. So the real question is: what can we do and what should we do with the current amount of resources we have? Personally I can also be involved in development of new (important in my opinion) enhancements to one of the greatest features in D, i.e. built in uinttesting. I'm even opting for changing the compiler if it's necessary (like expanding "-unittest" switch). But of course limiting to DRuntime only would be a step forward anyway. I wonder what Walter thinks about it now, because as far as I remember he was against build-in unittest improvements. Cheers, Piotrek
Dec 23 2016
parent reply Kagamin <spam here.lot> writes:
On Friday, 23 December 2016 at 16:25:13 UTC, Piotrek wrote:
 In result I have to accept small obstacles and go on. Otherwise 
 I wouldn't go anywhere.

 So the real question is: what can we do and what should we do 
 with the current amount of resources we have?
If you need a testing framework, try dunit https://wiki.dlang.org/Libraries_and_Frameworks#Unit_Testing_Framework
Dec 23 2016
parent Piotrek <piotrek unknownuniverse.net> writes:
On Friday, 23 December 2016 at 17:22:48 UTC, Kagamin wrote:
 On Friday, 23 December 2016 at 16:25:13 UTC, Piotrek wrote:
 In result I have to accept small obstacles and go on. 
 Otherwise I wouldn't go anywhere.

 So the real question is: what can we do and what should we do 
 with the current amount of resources we have?
If you need a testing framework, try dunit https://wiki.dlang.org/Libraries_and_Frameworks#Unit_Testing_Framework
No. I want to use build-in unit testing. Cheers, Piotrek
Dec 23 2016
prev sibling parent Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D
Subject: Re: ModuleInfo, factories, and unittesting
References: <o36etv$1dkf$1 digitalmars.com> <o3aogl$2imd$1 digitalmars.com>
 <o3bthc$2har$1 digitalmars.com> <o3bvee$2lku$1 digitalmars.com>
 <o3c4p0$2vqg$1 digitalmars.com>
In-Reply-To: <o3c4p0$2vqg$1 digitalmars.com>

--84cQE49KDbGbjOdVhuGfEtTRfIr7HIfq5
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 12/20/2016 10:36 PM, Walter Bright wrote:
 On 12/20/2016 11:05 AM, Dicebot wrote:
 Yes, pretty much. What ways do you have in mind? I am only aware of tw=
o:
 1) ModuleInfo
 2) https://dlang.org/spec/traits.html#getUnitTests
=20 =20 Put pointers to them in a special segment.
Oh, so you have meant "other ways can be implemented", not "other ways exist"? Sure. It does need to include qualified module names in that info though to preserve existing test runner functionality. --84cQE49KDbGbjOdVhuGfEtTRfIr7HIfq5--
Dec 21 2016