• Andrea Fontana (26/26) Jan 18 2012 I'm working on a complex project for my company.
• Trass3r (1/5) Jan 18 2012 ? dmd also uses ld to link.
• Andrea Fontana (2/8) Jan 18 2012
• Andrea Fontana (25/37) Jan 18 2012 Ok, here a test:
• Trass3r (4/7) Jan 18 2012 Well dmd test.d calls
• Andrea Fontana (19/27) Jan 18 2012 This param:
• Martin Nowak (3/30) Jan 18 2012 Do you have a lot of exceptions being thrown in your code?
• Andrea Fontana (5/46) Jan 18 2012 Not so much. Maybe inside check() of std.xml that check for xml
• Jacob Carlborg (4/33) Jan 18 2012 How about DMD passing --export-dynamic to GCC if -shared is passed to DM...
• Andrea Fontana (5/8) Jan 18 2012 I'm agree but it seems it's used to provide stack traces too (I just
• Marco Leise (5/7) Jan 19 2012 --export-dynamic is required for the symbol names in exception back trac...
• Jacob Carlborg (6/29) Jan 18 2012 Have you tried compiling and linking in one step with dmd? Just like thi...
• Martin Nowak (1/3) Jan 18 2012 Where can I get this particular file?
• Andrea Fontana (5/8) Jan 19 2012 I took a random medium-sized xml :)

Andrea Fontana <advmail katamail.com> writes:

I'm working on a complex project for my company.
It's a text-parser that handles complex/ambiguos word-based grammars. It
reads a set of xml files that define rules, build some structures
(trees, dictionaries, etc..) and then parses your phrases.  It works
fine.

I compile sources in this (simple) way:
dmd -c *.d   (three source files)

If i link objects with dmd execution of unit tests takes:

Parsing: 722 ms, 419 =CE=BCs, and 3 hnsecs

real	5m10.199s
user	5m9.771s
sys	0m0.236s


using gcc as linker:

Parsing: 425 ms, 677 =CE=BCs, and 8 hnsecs

real	0m13.919s
user	0m13.845s
sys	0m0.088s


unittest outputs are identical and execution time of test phrases has
similar speed but dmd version takes a lot of time parsing xml and
building trees and other structures.

I can't post company code, but i wonder if this is a known issue... If
not i'll try to write some code to reproduce this problem...

Configuration:
Ubuntu 11.10 64bit
DMD64 D Compiler v2.056
gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3)=20

Trass3r <un known.com> writes:

 Configuration:
 Ubuntu 11.10 64bit
 DMD64 D Compiler v2.056
 gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3)

? dmd also uses ld to link.

Andrea Fontana <advmail katamail.com> writes:

I can't understand why it's 22 times slower...

Il giorno mer, 18/01/2012 alle 14.54 +0100, Trass3r ha scritto:

 Configuration:
 Ubuntu 11.10 64bit
 DMD64 D Compiler v2.056
 gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3)

=20
 ? dmd also uses ld to link.

Andrea Fontana <advmail katamail.com> writes:

Ok, here a test:

test.d :

import std.xml;
import std.stdio;
import std.conv;

int main(string[] args)
{
        if (args.length !=3D 2)
        {
                writeln("Usage: ", args[0], " file.xml");
                return 0;
        }

        string content =3D to!string(std.file.read(args[1]));
       =20
        try{
                check(content);
        } catch (CheckException ex) { writeln("Exception: ", ex); }
        return 0;
}


dmd -c test.d && gcc test.o -lphobos2 -lrt -lpthread -o testgcc
vs
dmd -c test.d && dmd test.o -oftestd

try this code using a long xml file (tested here with a 2.0 mb list of
italian cities) ...

Il giorno mer, 18/01/2012 alle 15.27 +0100, Andrea Fontana ha scritto:

 I can't understand why it's 22 times slower...
=20
 Il giorno mer, 18/01/2012 alle 14.54 +0100, Trass3r ha scritto:=20
=20
 Configuration:
 Ubuntu 11.10 64bit
 DMD64 D Compiler v2.056
 gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3)

=20
 ? dmd also uses ld to link.

=20
=20

Trass3r <un known.com> writes:

 dmd -c test.d && gcc test.o -lphobos2 -lrt -lpthread -o testgcc
 vs
 dmd -c test.d && dmd test.o -oftestd

Well dmd test.d calls
gcc test.o -o test -m64 -Xlinker -L/dmd/linux/lib64 -Xlinker  
-L/dmd/linux/lib32 -Xlinker --no-warn-search-mismatch -Xlinker  
--export-dynamic -lphobos2 -lpthread -lm -lrt

Andrea Fontana <advmail katamail.com> writes:

This param:
--Xlinker --export-dynamic =20

slows down the entire app (check() std.xml function) by 29x on test.=20

Is it useful to add it on default config? Probably if you use external
dl, you should add it...

andrea ububox:~/src/xml-test$ dmd -c test.d && gcc test.o -o testgcc2
-m64 -Xlinker --export-dynamic -Xlinker --no-warn-search-mismatch
-lphobos2 -lpthread -lm -lrt
andrea ububox:~/src/xml-test$ time ./testgcc2 cities.xml=20

real	3m23.426s
user	3m22.901s
sys	0m0.276s

andrea ububox:~/src/xml-test$ dmd -c test.d && gcc test.o -o testgcc2
-m64 -Xlinker --no-warn-search-mismatch -lphobos2 -lpthread -lm -lrt
andrea ububox:~/src/xml-test$ time ./testgcc2 cities.xml=20

real	0m7.433s
user	0m7.316s
sys	0m0.112s

Il giorno mer, 18/01/2012 alle 15.43 +0100, Trass3r ha scritto:

 dmd -c test.d && gcc test.o -lphobos2 -lrt -lpthread -o testgcc
 vs
 dmd -c test.d && dmd test.o -oftestd

=20
 Well dmd test.d calls
 gcc test.o -o test -m64 -Xlinker -L/dmd/linux/lib64 -Xlinker =20
 -L/dmd/linux/lib32 -Xlinker --no-warn-search-mismatch -Xlinker =20
 --export-dynamic -lphobos2 -lpthread -lm -lrt

"Martin Nowak" <dawg dawgfoto.de> writes:

On Wed, 18 Jan 2012 16:00:05 +0100, Andrea Fontana <advmail katamail.com>  
wrote:

 This param:
 --Xlinker --export-dynamic

 slows down the entire app (check() std.xml function) by 29x on test.

 Is it useful to add it on default config? Probably if you use external
 dl, you should add it...

 andrea ububox:~/src/xml-test$ dmd -c test.d && gcc test.o -o testgcc2
 -m64 -Xlinker --export-dynamic -Xlinker --no-warn-search-mismatch
 -lphobos2 -lpthread -lm -lrt
 andrea ububox:~/src/xml-test$ time ./testgcc2 cities.xml

 real	3m23.426s
 user	3m22.901s
 sys	0m0.276s

 andrea ububox:~/src/xml-test$ dmd -c test.d && gcc test.o -o testgcc2
 -m64 -Xlinker --no-warn-search-mismatch -lphobos2 -lpthread -lm -lrt
 andrea ububox:~/src/xml-test$ time ./testgcc2 cities.xml

 real	0m7.433s
 user	0m7.316s
 sys	0m0.112s

 Il giorno mer, 18/01/2012 alle 15.43 +0100, Trass3r ha scritto:

 dmd -c test.d && gcc test.o -lphobos2 -lrt -lpthread -o testgcc
 vs
 dmd -c test.d && dmd test.o -oftestd

 Well dmd test.d calls
 gcc test.o -o test -m64 -Xlinker -L/dmd/linux/lib64 -Xlinker
 -L/dmd/linux/lib32 -Xlinker --no-warn-search-mismatch -Xlinker
 --export-dynamic -lphobos2 -lpthread -lm -lrt


Do you have a lot of exceptions being thrown in your code?

Andrea Fontana <advmail katamail.com> writes:

Not so much. Maybe inside check() of std.xml that check for xml
correctness and throw exception. It's the function that slow down the
entire app.

Il giorno mer, 18/01/2012 alle 16.26 +0100, Martin Nowak ha scritto:

 On Wed, 18 Jan 2012 16:00:05 +0100, Andrea Fontana <advmail katamail.com>=

 =20
 wrote:
=20
 This param:
 --Xlinker --export-dynamic

 slows down the entire app (check() std.xml function) by 29x on test.

 Is it useful to add it on default config? Probably if you use external
 dl, you should add it...

 andrea ububox:~/src/xml-test$ dmd -c test.d && gcc test.o -o testgcc2
 -m64 -Xlinker --export-dynamic -Xlinker --no-warn-search-mismatch
 -lphobos2 -lpthread -lm -lrt
 andrea ububox:~/src/xml-test$ time ./testgcc2 cities.xml

 real	3m23.426s
 user	3m22.901s
 sys	0m0.276s

 andrea ububox:~/src/xml-test$ dmd -c test.d && gcc test.o -o testgcc2
 -m64 -Xlinker --no-warn-search-mismatch -lphobos2 -lpthread -lm -lrt
 andrea ububox:~/src/xml-test$ time ./testgcc2 cities.xml

 real	0m7.433s
 user	0m7.316s
 sys	0m0.112s

 Il giorno mer, 18/01/2012 alle 15.43 +0100, Trass3r ha scritto:

 dmd -c test.d && gcc test.o -lphobos2 -lrt -lpthread -o testgcc
 vs
 dmd -c test.d && dmd test.o -oftestd

 Well dmd test.d calls
 gcc test.o -o test -m64 -Xlinker -L/dmd/linux/lib64 -Xlinker
 -L/dmd/linux/lib32 -Xlinker --no-warn-search-mismatch -Xlinker
 --export-dynamic -lphobos2 -lpthread -lm -lrt


 Do you have a lot of exceptions being thrown in your code?

Jacob Carlborg <doob me.com> writes:

On 2012-01-18 16:00, Andrea Fontana wrote:
 This param:
 --Xlinker --export-dynamic

 slows down the entire app (check() std.xml function) by 29x on test.

 Is it useful to add it on default config? Probably if you use external
 dl, you should add it...

 andrea ububox <mailto:andrea ububox>:~/src/xml-test$ dmd -c test.d &&
 gcc test.o -o testgcc2 -m64 -Xlinker --export-dynamic -Xlinker
 --no-warn-search-mismatch -lphobos2 -lpthread -lm -lrt
 andrea ububox <mailto:andrea ububox>:~/src/xml-test$ time ./testgcc2
 cities.xml

 real 3m23.426s
 user 3m22.901s
 sys 0m0.276s

 andrea ububox:~/src/xml-test$ dmd -c test.d && gcc test.o -o testgcc2
 -m64 -Xlinker --no-warn-search-mismatch -lphobos2 -lpthread -lm -lrt
 andrea ububox <mailto:andrea ububox>:~/src/xml-test$ time ./testgcc2
 cities.xml

 real 0m7.433s
 user 0m7.316s
 sys 0m0.112s

 Il giorno mer, 18/01/2012 alle 15.43 +0100, Trass3r ha scritto:
  dmd -c test.d&&  gcc test.o -lphobos2 -lrt -lpthread -o testgcc
  vs
  dmd -c test.d&&  dmd test.o -oftestd

 Well dmd test.d calls
 gcc test.o -o test -m64 -Xlinker -L/dmd/linux/lib64 -Xlinker
 -L/dmd/linux/lib32 -Xlinker --no-warn-search-mismatch -Xlinker
 --export-dynamic -lphobos2 -lpthread -lm -lrt


How about DMD passing --export-dynamic to GCC if -shared is passed to DMD?

-- 
/Jacob Carlborg

Andrea Fontana <advmail katamail.com> writes:

I'm agree but it seems it's used to provide stack traces too (I just
read it on an old post). So maybe if -shared or if we're in debug
mode...

Il giorno mer, 18/01/2012 alle 16.35 +0100, Jacob Carlborg ha scritto:

=20
 How about DMD passing --export-dynamic to GCC if -shared is passed to DMD=

?
=20

"Marco Leise" <Marco.Leise gmx.de> writes:

Am 18.01.2012, 16:35 Uhr, schrieb Jacob Carlborg <doob me.com>:

 How about DMD passing --export-dynamic to GCC if -shared is passed to  
 DMD?

--export-dynamic is required for the symbol names in exception back traces  
and comes from dmd.conf. That said, you can remove it there and enable it  
only for debug builds inside a Makefile to still get meaningful messages  
for failed asserts (back traces with files/line numbers).

Jacob Carlborg <doob me.com> writes:

On 2012-01-18 15:37, Andrea Fontana wrote:
 Ok, here a test:

 test.d :

 import std.xml;
 import std.stdio;
 import std.conv;

 int main(string[] args)
 {
 if (args.length != 2)
 {
 writeln("Usage: ", args[0], " file.xml");
 return 0;
 }

 string content = to!string(std.file.read(args[1]));

 try{
 check(content);
 } catch (CheckException ex) { writeln("Exception: ", ex); }
 return 0;
 }


 dmd -c test.d && gcc test.o -lphobos2 -lrt -lpthread -o testgcc
 vs
 dmd -c test.d && dmd test.o -oftestd

 try this code using a long xml file (tested here with a 2.0 mb list of
 italian cities) ...

Have you tried compiling and linking in one step with dmd? Just like this:

dmd test.d

BTW, you can pass "-v2 to dmd to see which flags it uses when invoking gcc.


-- 
/Jacob Carlborg

"Martin Nowak" <dawg dawgfoto.de> writes:

 try this code using a long xml file (tested here with a 2.0 mb list of
 italian cities) ...

Where can I get this particular file?

Andrea Fontana <advmail katamail.com> writes:

I took a random medium-sized xml :)

By the way for this test I downloaded it:
http://www.ollie10.it/file.axd?file=3D2010%2f4%
2fDatabase_Comuni_Italiani_2010.zip (folder /xml/Cities.xml)

Il giorno mer, 18/01/2012 alle 18.28 +0100, Martin Nowak ha scritto:

 try this code using a long xml file (tested here with a 2.0 mb list of
 italian cities) ...

 Where can I get this particular file?