• JG (5/5) Sep 02 2021 Hi,
• jfondren (3/9) Sep 02 2021 Possibly it's not a compiler error at all but a name-mangling
• JG (7/17) Sep 03 2021 Thanks for the suggestion.
• Vladimir Panteleev (16/22) Sep 02 2021 One way to get a very rough estimate is to take the square of the
• Vladimir Panteleev (6/10) Sep 02 2021 I meant the square of the size of the respective directory.
• JG (9/19) Sep 03 2021 Thanks for the information.
• JG (14/36) Sep 03 2021 I tried again. What am I doing wrong?
• JG (2/17) Sep 03 2021 First line should be: cp -R source ~/tmp/source
• Vladimir Panteleev (16/29) Sep 03 2021 One potential source of trouble is using Dub. Dub doesn't play
• Andre Pany (11/17) Sep 04 2021 Hi,
• Vladimir Panteleev (4/8) Sep 04 2021 Overly generic test conditions would be more likely to cause
• JG (10/18) Sep 04 2021 In the end I did the reduction by hand and discovered that the
• JG (11/31) Sep 04 2021 As a small comment regarding dub. I can't help wondering if it
• Mike Parker (5/15) Sep 04 2021 You should be able to do that now with "sourceFiles" and
• JG (2/8) Sep 04 2021 I didn't know about this. Thanks you for letting me know.

JG <JG somewhere.com> writes:

Hi,

We hit a linking error (after upgrading to dub 1.26.0). I thought 
I would try to use dustmite to create a reduced error test case. 
One week later it is still running (depth 22). I don't suppose 
there is anyway of determining when it will finish?

jfondren <julian.fondren gmail.com> writes:

On Thursday, 2 September 2021 at 11:04:12 UTC, JG wrote:
 Hi,

 We hit a linking error (after upgrading to dub 1.26.0). I 
 thought I would try to use dustmite to create a reduced error 
 test case. One week later it is still running (depth 22). I 
 don't suppose there is anyway of determining when it will 
 finish?

Possibly it's not a compiler error at all but a name-mangling 
change combined with some stale objects in your build.

JG <JG somewhere.com> writes:

On Thursday, 2 September 2021 at 11:19:55 UTC, jfondren wrote:
 On Thursday, 2 September 2021 at 11:04:12 UTC, JG wrote:
 Hi,

 We hit a linking error (after upgrading to dub 1.26.0). I 
 thought I would try to use dustmite to create a reduced error 
 test case. One week later it is still running (depth 22). I 
 don't suppose there is anyway of determining when it will 
 finish?

 Possibly it's not a compiler error at all but a name-mangling 
 change combined with some stale objects in your build.

Thanks for the suggestion.

I thought I was doing this. What I did was run:

dub clean --all-packages
dub build --force

Then I am getting twelve linking errors. Of the form:
......  error: undefined reference to .....

Vladimir Panteleev <thecybershadow.lists gmail.com> writes:

On Thursday, 2 September 2021 at 11:04:12 UTC, JG wrote:
 Hi,

 We hit a linking error (after upgrading to dub 1.26.0). I 
 thought I would try to use dustmite to create a reduced error 
 test case. One week later it is still running (depth 22). I 
 don't suppose there is anyway of determining when it will 
 finish?

One way to get a very rough estimate is to take the square of the 
current reduction (.reduced directory), and divide it by the 
square of the original source.

For best performance, make sure that:

1) You are using the latest version from 
https://github.com/CyberShadow/DustMite
2) DustMite itself is built in release mode
3) You are using the -j switch to enable parallelism
4) You are running the reduction on a RAM disk or equivalent

If you don't want to risk losing progress by putting everything 
(including the result directory) on a RAM disk, the latest 
bleeding-edge version on the "next" branch introduces a 
--temp-dir option.

DustMite reductions can be stopped and resumed by pointing 
DustMite at the latest .reduced directory from a previous run.

Vladimir Panteleev <thecybershadow.lists gmail.com> writes:

On Thursday, 2 September 2021 at 11:20:18 UTC, Vladimir Panteleev 
wrote:
 One way to get a very rough estimate is to take the square of 
 the current reduction (.reduced directory), and divide it by 
 the square of the original source.

I meant the square of the size of the respective directory. 
(bytes / LOC / SLOC...)

 One week later it is still running (depth 22).


If you are still on the first iteration, you may also try 
switching to the "careful" strategy.

JG <JG somewhere.com> writes:

On Thursday, 2 September 2021 at 17:56:54 UTC, Vladimir Panteleev 
wrote:
 On Thursday, 2 September 2021 at 11:20:18 UTC, Vladimir 
 Panteleev wrote:
 One way to get a very rough estimate is to take the square of 
 the current reduction (.reduced directory), and divide it by 
 the square of the original source.

 I meant the square of the size of the respective directory. 
 (bytes / LOC / SLOC...)

 One week later it is still running (depth 22).


 If you are still on the first iteration, you may also try 
 switching to the "careful" strategy.

Thanks for the information.

I stopped dustMite and realized I must have done something wrong 
since the reduced test case doesn't reproduce the problem.

I would really like to try and produce a reduced test case of 
this problem. However having spent three or four hours trying to 
figure out what I am doing wrong I think I will have to stop at 
this point.

JG <JG somewhere.com> writes:

On Friday, 3 September 2021 at 19:56:30 UTC, JG wrote:
 On Thursday, 2 September 2021 at 17:56:54 UTC, Vladimir 
 Panteleev wrote:
 On Thursday, 2 September 2021 at 11:20:18 UTC, Vladimir 
 Panteleev wrote:
 One way to get a very rough estimate is to take the square of 
 the current reduction (.reduced directory), and divide it by 
 the square of the original source.

 I meant the square of the size of the respective directory. 
 (bytes / LOC / SLOC...)

 One week later it is still running (depth 22).


 If you are still on the first iteration, you may also try 
 switching to the "careful" strategy.

 Thanks for the information.

 I stopped dustMite and realized I must have done something 
 wrong since the reduced test case doesn't reproduce the problem.

 I would really like to try and produce a reduced test case of 
 this problem. However having spent three or four hours trying 
 to figure out what I am doing wrong I think I will have to stop 
 at this point.

I tried again. What am I doing wrong?

cp source ~/tmp/source
cd ~/tmp/source
dub build --config prog1 2>&1 | grep "collect2: error: ld 
returned 1 exit status"
echo $? #produces 0
find . -name *.o -delete
~/d/DustMite/dustmite -j ./ 'dub build --config prog1 2>&1 | grep 
"collect2: error: ld returned 1 exit status"'
cd ../source.reduced
dub build --config prog1 2>&1 | grep "collect2: error: ld 
returned 1 exit status"
echo $? #produces 1

JG <JG somewhere.com> writes:

On Saturday, 4 September 2021 at 06:18:52 UTC, JG wrote:
 On Friday, 3 September 2021 at 19:56:30 UTC, JG wrote:
 [...]

 I tried again. What am I doing wrong?

 cp source ~/tmp/source
 cd ~/tmp/source
 dub build --config prog1 2>&1 | grep "collect2: error: ld 
 returned 1 exit status"
 echo $? #produces 0
 find . -name *.o -delete
 ~/d/DustMite/dustmite -j ./ 'dub build --config prog1 2>&1 | 
 grep "collect2: error: ld returned 1 exit status"'
 cd ../source.reduced
 dub build --config prog1 2>&1 | grep "collect2: error: ld 
 returned 1 exit status"
 echo $? #produces 1

First line should be: cp -R source ~/tmp/source

Vladimir Panteleev <thecybershadow.lists gmail.com> writes:

On Saturday, 4 September 2021 at 06:18:52 UTC, JG wrote:
 I tried again. What am I doing wrong?

 cp source ~/tmp/source
 cd ~/tmp/source
 dub build --config prog1 2>&1 | grep "collect2: error: ld 
 returned 1 exit status"
 echo $? #produces 0
 find . -name *.o -delete
 ~/d/DustMite/dustmite -j ./ 'dub build --config prog1 2>&1 | 
 grep "collect2: error: ld returned 1 exit status"'
 cd ../source.reduced
 dub build --config prog1 2>&1 | grep "collect2: error: ld 
 returned 1 exit status"
 echo $? #produces 1

One potential source of trouble is using Dub. Dub doesn't play 
well with tools such as DustMite, because it keeps state outside 
of the current directory. In some cases, it even writes to that 
state, which works very poorly especially with `-j`. Dub has a 
built-in "dustmite" command, though it is a bit limited - you 
could use `dub -v` to get the compiler and linker commands that 
Dub runs, and run them directly from your test command.

Some bugs do not reproduce 100% of the time, and are random in 
nature, due to depending on memory layout or thread timing. Try 
running the test command a number of times and see if that 
affects the result.

Regardless, your workflow looks OK from a high level - the 
source.reduced directory should indeed contain the contents of a 
directory for which the test command did return 0, for one reason 
or another.

Andre Pany <andre s-e-a-p.de> writes:

On Thursday, 2 September 2021 at 11:04:12 UTC, JG wrote:
 Hi,

 We hit a linking error (after upgrading to dub 1.26.0). I 
 thought I would try to use dustmite to create a reduced error 
 test case. One week later it is still running (depth 22). I 
 don't suppose there is anyway of determining when it will 
 finish?

Hi,
The Dustmite condition you are using seems very generic, 
therefore this error message can be triggered by various code 
constellations. This might be the reasons why Dustmite is running 
so long.

You might delete all fetched dub packages in your home directory 
and also delete .dub in your project directory. Hopefully this 
already solves the issue.

Kind regards
Andre

Vladimir Panteleev <thecybershadow.lists gmail.com> writes:

On Saturday, 4 September 2021 at 07:38:34 UTC, Andre Pany wrote:
 The Dustmite condition you are using seems very generic, 
 therefore this error message can be triggered by various code 
 constellations. This might be the reasons why Dustmite is 
 running so long.

Overly generic test conditions would be more likely to cause 
DustMite to finish very quickly (and produce the wrong result), 
rather than the other way around.

JG <JG somewhere.com> writes:

On Saturday, 4 September 2021 at 07:40:07 UTC, Vladimir Panteleev 
wrote:
 On Saturday, 4 September 2021 at 07:38:34 UTC, Andre Pany wrote:
 The Dustmite condition you are using seems very generic, 
 therefore this error message can be triggered by various code 
 constellations. This might be the reasons why Dustmite is 
 running so long.

 Overly generic test conditions would be more likely to cause 
 DustMite to finish very quickly (and produce the wrong result), 
 rather than the other way around.

In the end I did the reduction by hand and discovered that the 
problem is that the behaviour of dub changed. It seems to now 
exclude the mainSourceFile of other configurations in the build 
(which I guess shouldn't matter) except we had an old 
configuration which after some renaming had a wrong 
mainSourceFile which was needed for the build of the 
configuration in question.

Thanks for the suggestions.

JG <JG somewhere.com> writes:

On Saturday, 4 September 2021 at 08:05:16 UTC, JG wrote:
 On Saturday, 4 September 2021 at 07:40:07 UTC, Vladimir 
 Panteleev wrote:
 On Saturday, 4 September 2021 at 07:38:34 UTC, Andre Pany 
 wrote:
 The Dustmite condition you are using seems very generic, 
 therefore this error message can be triggered by various code 
 constellations. This might be the reasons why Dustmite is 
 running so long.

 Overly generic test conditions would be more likely to cause 
 DustMite to finish very quickly (and produce the wrong 
 result), rather than the other way around.

 In the end I did the reduction by hand and discovered that the 
 problem is that the behaviour of dub changed. It seems to now 
 exclude the mainSourceFile of other configurations in the build 
 (which I guess shouldn't matter) except we had an old 
 configuration which after some renaming had a wrong 
 mainSourceFile which was needed for the build of the 
 configuration in question.

 Thanks for the suggestions.

As a small comment regarding dub. I can't help wondering if it 
really the best idea for each configuration to include everything 
by default and then have to exclude things? This means that when 
you add another configuration and source files for it you very 
often have to modify all other existing ones. If instead you 
choose what to include this wouldn't happen. Wild cards support 
could be added for included files to make projects with a single 
configuration just as simple as now. Just some thoughts. (I feel 
the same way regarding gitignore, I would rather have the 
opposite.)

Mike Parker <aldacron gmail.com> writes:

On Saturday, 4 September 2021 at 08:19:53 UTC, JG wrote:

 As a small comment regarding dub. I can't help wondering if it 
 really the best idea for each configuration to include 
 everything by default and then have to exclude things? This 
 means that when you add another configuration and source files 
 for it you very often have to modify all other existing ones. 
 If instead you choose what to include this wouldn't happen. 
 Wild cards support could be added for included files to make 
 projects with a single configuration just as simple as now. 
 Just some thoughts. (I feel the same way regarding gitignore, I 
 would rather have the opposite.)

You should be able to do that now with "sourceFiles" and 
"sourcePaths". Just avoid the default "source" or "src" 
directories and specify the paths and/or files you want for each 
configuration.

JG <someone somewhere.com> writes:

On Saturday, 4 September 2021 at 08:54:31 UTC, Mike Parker wrote:
 On Saturday, 4 September 2021 at 08:19:53 UTC, JG wrote:

 [...]

 You should be able to do that now with "sourceFiles" and 
 "sourcePaths". Just avoid the default "source" or "src" 
 directories and specify the paths and/or files you want for 
 each configuration.

I didn't know about this. Thanks you for letting me know.