www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 8596] New: Indeterministic assertion failure in rehash

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596

           Summary: Indeterministic assertion failure in rehash
           Product: D
           Version: D2
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P2
         Component: DMD
        AssignedTo: nobody puremagic.com
        ReportedBy: timon.gehr gmx.ch



DMD 2.060

The following assertion failure occurs indeterministically:

dmd: ../ztc/aa.c:423: void AArray::rehash_x(aaA*, aaA**, size_t): Assertion `0'
failed.

The frequency seems to increase as project size grows.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Aug 28 2012
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596


deadalnix <deadalnix gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |deadalnix gmail.com



It tend to become a serious problem to me.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 02 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596






 It tend to become a serious problem to me.


+1.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 02 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596


Walter Bright <bugzilla digitalmars.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bugzilla digitalmars.com



11:50:58 PST ---
The assert happens when more than one item in the hash table has the same key.
A hash table may only have one item per key.

It would be helpful if you could run it under linux or osx, compile dmd with
-g, and get a stack trace.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 12 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596


Walter Bright <bugzilla digitalmars.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|normal                      |major


-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 12 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596


deadalnix <deadalnix gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|major                       |normal





 The assert happens when more than one item in the hash table has the same key.
 A hash table may only have one item per key.
 
 It would be helpful if you could run it under linux or osx, compile dmd with
 -g, and get a stack trace.


I fail to trigger the error when using gdb. Seems like gdb is influencing on
what does trigger the error.

The error trigger as well when dmd is compiled with -g .

Looks like a race condition.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 12 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596




12:39:20 PST ---
dmd isn't multithreaded, so it could not be a race condition.

It does, however, sound like memory corruption.

Can you run it under valgrind?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 12 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596






 dmd isn't multithreaded, so it could not be a race condition.
 
 It does, however, sound like memory corruption.
 
 Can you run it under valgrind?


Already tryed, but I don't have enough RAM on my machine to do so. Consider
that compiling the project require more than 2.2Gb of RAM without valgrind.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 12 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596




12:51:20 PST ---
If you're running the 64 bit dmd, shouldn't it be able to use far more virtual
memory (very slowly)?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 12 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596


Alex Rønne Petersen <alex lycus.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |alex lycus.org



CET ---
Well, only if paging is enabled (i.e. there is a swap partition in use). I know
I set my systems up without paging because I practically never need it.

 deadalnix How much RAM does your system have?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 12 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596






 Well, only if paging is enabled (i.e. there is a swap partition in use). I know
 I set my systems up without paging because I practically never need it.
 
  deadalnix How much RAM does your system have?


I have 4Gb. Minus what the system uses plus some programs running, it is
already swapping when dmd uses 2.2Gb.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 12 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596








 Well, only if paging is enabled (i.e. there is a swap partition in use). I know
 I set my systems up without paging because I practically never need it.
 
  deadalnix How much RAM does your system have?


 
 I have 4Gb. Minus what the system uses plus some programs running, it is
 already swapping when dmd uses 2.2Gb.


OK, I did ran valgrind into a terminal without any graphical interface. I hope
I'll not run out of swap because of the memory I won disabling the graphical
interface.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 12 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596




Created an attachment (id=1160)
valgrind output

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 12 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596




16:56:17 PST ---


 Created an attachment (id=1160) [details]
 valgrind output


Fixed the valgrind reported issue. It's definitely a corruption bug in the aa.c
code. Testing now.

If you want to try it out:

==============================
diff aa.bak aa.c
72c72
<                 delete en;
---

                 delete [] en;


79c79
<         delete en;
---

         delete [] en;


==============================

I love valgrind.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 12 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596




17:09:08 PST ---
https://github.com/D-Programming-Language/dmd/commit/80884506df7a020e879ba3adda5a98d0465e7164

I won't mark it fixed until you guys can verify, as I don't have your test
code.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 12 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596






 https://github.com/D-Programming-Language/dmd/commit/80884506df7a020e879ba3adda5a98d0465e7164
 
 I won't mark it fixed until you guys can verify, as I don't have your test
 code.


Bad news : just tested the patch and got 6 out of 10 compilation failed on this
assert.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 12 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596




17:30:24 PST ---


 
 Bad news : just tested the patch and got 6 out of 10 compilation failed on this
 assert.


What do you mean? Dmd won't compile?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 12 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596








 
 Bad news : just tested the patch and got 6 out of 10 compilation failed on this
 assert.


 
 What do you mean? Dmd won't compile?


No, dmd compile :D But my program didn't using that patched dmd 6 time out of
10.

It means that the issue remains.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 12 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596


bearophile_hugs eml.cc changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bearophile_hugs eml.cc






 It means that the issue remains.


Then I suggest to run it again in Valgrind :-) Maybe there are other bugs to
fix it spots.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 12 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596







 

 It means that the issue remains.


 
 Then I suggest to run it again in Valgrind :-) Maybe there are other bugs to
 fix it spots.


I did, but found nothing.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 13 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596




13:32:12 PST ---
Ok, the next step is to compile your app without -g. The reason is because hash
tables are used in the dwarf debug generation, and I want to see if that one is
the problem or other uses.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 13 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596






 Ok, the next step is to compile your app without -g. The reason is because hash
 tables are used in the dwarf debug generation, and I want to see if that one is
 the problem or other uses.


Tested with flags : -m64 -w -debug -unittest (removing the -gc flag I usually
use).

I « successfully » triggered the error as well.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 14 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596




16:59:25 PST ---
I've tried various schemes to induce it to fail, and done a code review. I
can't find anything wrong. I need a test case.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 14 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596






 I've tried various schemes to induce it to fail, and done a code review. I
 can't find anything wrong. I need a test case.


Sorry for being unavailable the past few days, I was moving from France to USA.

I can trigger the error on a regular basis with the following codebase :

git clone git://github.com/deadalnix/SDC.git
cd SDC
git checkout aa_assert
make

the program is made to be compiled on linux. My computer is a dual core athlon
with 4Gb of RAM.

I can't come up with a simple test case that trigger the error, because it
seems that size matter here.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 25 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596




04:58:03 PST ---
When I try your instructions, I get:


make
~/cbx/mars/dmd -ofbin/sdc src/sdc/*.d src/sdc/ast/*.d src/sdc/pass/*.d
src/d/ast/*.d src/d/backend/*.d src/d/parser/*.d src/d/pass/*.d src/util/*.d
src/etc/linux/*.d import/llvm/c/target.d -m64 -w -debug -gc -unittest -Iimport
-L-L`llvm-config-3.1 --libdir` `llvm-config-3.1 --libs | sed 's/-l/-L-l/g'`
-L-lstdc++ -L-ldl -L-lffi
/bin/sh: llvm-config-3.1: not found
/bin/sh: llvm-config-3.1: not found
src/sdc/compilererror.d(188): Warning: statement is not reachable
dmd: func.c:1200: virtual void FuncDeclaration::semantic3(Scope*): Assertion
`type == f' failed.
Aborted
make: *** [bin/sdc] Error 134


which is not the error you reported.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Dec 10 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596


Rene Zwanenburg <renezwanenburg gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |renezwanenburg gmail.com



18:25:08 PDT ---
Running into this issue since today. I'm developing on a 64 bit windows machine
targeting 32 bit, production machine is running 64 bit debian. Using DMD 2.062
on both boxes.

The bug pops up at the strangest moments. I can confirm it's indeterministic,
but so far the production box never failed to build. That said, I don't build
very often on production, esp. if the build failed on the dev box ;). For the
sake of completeness I've just tried compiling a version of the project on the
production box which almost always fails on the dev box. So far it keeps
succeeding.

On the windows box I recently flipped the LARGE_ADDRESS_AWARE bit on dmd.exe to
work around issue 6498, perhaps this has something to do with it?

As an indication of project size, the project is medium sized: vibe.d + 1937
lines, but it's very CTFE heavy: we're using vibe.d's diet template parser to
generate 21 rather large web pages. Compilation requires a little over two GB
of memory.

Is there any additional info I can provide?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 01 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596


Maxim Fomin <maxim maxim-fomin.ru> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |maxim maxim-fomin.ru



---
I think somebody should provide a (snapshot of) project to compile to
investigate the problem, deadalnix's link is outdated and guessing based on
valgrind output is not a very good idea.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 02 2013
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=8596




07:24:10 PDT ---
Yeah I was afraid of that :)

I'd have to take it up with the PHB's, not sure what the answer will be. TBH I
doubt I can make the code public. Not that there's anything of value in it, but
you know how it is...

I'll report back when I've got an answer.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 02 2013