www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - questions about NetBSD port

reply Nikolay <sibnick gmail.com> writes:
I am porting LDC/phobos/druntime to NetBSD. Currently my patch is 
merged into LDC master. I have several questions about 
phobos/druntime and general workflow.
As I can understand I should prepare pull requests for 
phobos/druntime master branches. LDC team will merge/cherry-pick 
changes into ldc branch from master later. Is it correct 
workflow?  Because it means that I can’t check my patch: there is 
no dmd compiler for NetBSD + phobos/druntime master branches.

Also I have a set of issues with real (long double) type. Several 
functions absent in NetBSD for real type (more specific - they 
are just aliases for functions with double). Other functions 
return less accurate values.
I have to disable a couple unit tests for NetBSD.
E.g. std/conv.d:2835 assert(to!string(r) == 
to!string(real.min_normal))
see related question:  
http://stackoverflow.com/questions/35090322/netbsd-long-double-trouble
But also there are a set of unit tests where I have to reduce 
accuracy. This problem affects std/internal/math/gammafunction.d, 
std/math.d, std/numeric.d, std/complex.d
E.g.  std/complex.d:792
assert(sin(complex(2.0L, 0)) == std.math.sin(2.0L));
I replace with:
assert(feqrel(sin(complex(2.0L, 0)).re, std.math.sin(2.0L)) > 
real.mant_dig-10);
My question is: should I wrap such changes with version(NetBSD) 
statement or it is acceptable reduce accuracy for all platforms?

PS
You can look to my code here (netbsd_patch branch):
https://github.com/nrTQgc/druntime
https://github.com/nrTQgc/phobos
Feb 02 2016
next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Tuesday, 2 February 2016 at 18:45:20 UTC, Nikolay wrote:

 I am porting LDC/phobos/druntime to NetBSD. Currently my patch 
 is merged into LDC master. I have several questions about 
 phobos/druntime and general workflow.
 As I can understand I should prepare pull requests for 
 phobos/druntime master branches. LDC team will 
 merge/cherry-pick changes into ldc branch from master later. Is 
 it correct workflow?  Because it means that I can’t check my 
 patch: there is no dmd compiler for NetBSD + phobos/druntime 
 master branches.


You're probably better off porting dmd 2.068 first (as it's the 
last dmd written wholly in C++), using it to compile dmd git 
master on NetBSD, then porting druntime and phobos master.  
Porting dmd to NetBSD/x86 is likely easy: simply follow what the 
other BSDs have done, as you did for ldc.


 Also I have a set of issues with real (long double) type. 
 Several functions absent in NetBSD for real type (more specific 
 - they are just aliases for functions with double). Other 
 functions return less accurate values.
 I have to disable a couple unit tests for NetBSD.
 E.g. std/conv.d:2835 assert(to!string(r) == 
 to!string(real.min_normal))
 see related question:  
 http://stackoverflow.com/questions/35090322/netbsd-long-double-trouble
 But also there are a set of unit tests where I have to reduce 
 accuracy. This problem affects 
 std/internal/math/gammafunction.d, std/math.d, std/numeric.d, 
 std/complex.d
 E.g.  std/complex.d:792
 assert(sin(complex(2.0L, 0)) == std.math.sin(2.0L));
 I replace with:
 assert(feqrel(sin(complex(2.0L, 0)).re, std.math.sin(2.0L)) > 
 real.mant_dig-10);
 My question is: should I wrap such changes with version(NetBSD) 
 statement or it is acceptable reduce accuracy for all platforms?


This is the same issue Win64 and Android have had.  You should 
already see version blocks for those in some places, you can do 
the same.  Take a look at my Android patches for similar tweaks 
that are still unmerged:

https://gist.github.com/joakim-noah/d936d6a339426ad1fac3
https://gist.github.com/joakim-noah/5c03801fa6c59b1e90df
Feb 02 2016
next sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 3 Feb 2016 7:30 am, "Joakim via Digitalmars-d" <
digitalmars-d puremagic.com> wrote:

 On Tuesday, 2 February 2016 at 18:45:20 UTC, Nikolay wrote:

 I am porting LDC/phobos/druntime to NetBSD. Currently my patch is merged




into LDC master. I have several questions about phobos/druntime and general
workflow.

 As I can understand I should prepare pull requests for phobos/druntime




master branches. LDC team will merge/cherry-pick changes into ldc branch
from master later. Is it correct workflow?  Because it means that I can=E2=
=80=99t
check my patch: there is no dmd compiler for NetBSD + phobos/druntime
master branches.

 You're probably better off porting dmd 2.068 first (as it's the last dmd


written wholly in C++), using it to compile dmd git master on NetBSD, then
porting druntime and phobos master.  Porting dmd to NetBSD/x86 is likely
easy: simply follow what the other BSDs have done, as you did for ldc.


I doubt it. Using ldc/gdc usually means that it's just the library you need
to play with, not the compiler.



 Feb 02 2016



  parent reply Joakim <dlang joakim.fea.st>  writes:


On Wednesday, 3 February 2016 at 07:01:10 UTC, Iain Buclaw wrote:

 On 3 Feb 2016 7:30 am, "Joakim via Digitalmars-d" < 
 digitalmars-d puremagic.com> wrote:

 On Tuesday, 2 February 2016 at 18:45:20 UTC, Nikolay wrote:

 I am porting LDC/phobos/druntime to NetBSD. Currently my 
 patch is merged




 into LDC master. I have several questions about phobos/druntime 
 and general workflow.

 As I can understand I should prepare pull requests for 
 phobos/druntime




 master branches. LDC team will merge/cherry-pick changes into 
 ldc branch from master later. Is it correct workflow?  Because 
 it means that I can’t check my patch: there is no dmd compiler 
 for NetBSD + phobos/druntime master branches.

 You're probably better off porting dmd 2.068 first (as it's 
 the last dmd


 written wholly in C++), using it to compile dmd git master on 
 NetBSD, then porting druntime and phobos master.  Porting dmd 
 to NetBSD/x86 is likely easy: simply follow what the other BSDs 
 have done, as you did for ldc.


 I doubt it. Using ldc/gdc usually means that it's just the 
 library you need to play with, not the compiler.


The problem is that ldc/gdc are not on 2.069, let alone the 
just-released 2.070.  So as he said, if he really wants to test 
his druntime/phobos PRs against master _on_ NetBSD, he has no 
choice but to port dmd.  I haven't had to do so myself, but 
looking at how it's been done for other BSDs, it looks fairly 
easy to me.



 Feb 02 2016



  parent  Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 3 Feb 2016 8:35 am, "Joakim via Digitalmars-d" <
digitalmars-d puremagic.com> wrote:

 On Wednesday, 3 February 2016 at 07:01:10 UTC, Iain Buclaw wrote:

 On 3 Feb 2016 7:30 am, "Joakim via Digitalmars-d" <




digitalmars-d puremagic.com> wrote:

 On Tuesday, 2 February 2016 at 18:45:20 UTC, Nikolay wrote:

 I am porting LDC/phobos/druntime to NetBSD. Currently my patch is








merged

 into LDC master. I have several questions about phobos/druntime and




general workflow.

 As I can understand I should prepare pull requests for phobos/druntime




 master branches. LDC team will merge/cherry-pick changes into ldc branch




from master later. Is it correct workflow?  Because it means that I can=E2=
=80=99t
check my patch: there is no dmd compiler for NetBSD + phobos/druntime
master branches.

 You're probably better off porting dmd 2.068 first (as it's the last dm=






d

 written wholly in C++), using it to compile dmd git master on NetBSD,




then porting druntime and phobos master.  Porting dmd to NetBSD/x86 is
likely easy: simply follow what the other BSDs have done, as you did for
ldc.


 I doubt it. Using ldc/gdc usually means that it's just the library you




need to play with, not the compiler.

 The problem is that ldc/gdc are not on 2.069, let alone the just-released


2.070.  So as he said, if he really wants to test his druntime/phobos PRs
against master _on_ NetBSD, he has no choice but to port dmd.  I haven't
had to do so myself, but looking at how it's been done for other BSDs, it
looks fairly easy to me.

Druntime doesn't change all that much, so it's trivial to rebase to
master.  As for phobos, the only parts that matter tend to be std.math and
co.

Maybe there are some places like std.file too, but I've likely been spoiled
by using version(GNU) in places where DMD chooses to have a version branch
for each platform.  :-)



 Feb 02 2016



prev sibling next sibling parent reply Nikolay <sibnick gmail.com>  writes:


On Wednesday, 3 February 2016 at 06:25:49 UTC, Joakim wrote:

 On Tuesday, 2 February 2016 at 18:45:20 UTC, Nikolay wrote:

 You're probably better off porting dmd 2.068 first (as it's the 
 last dmd written wholly in C++), using it to compile dmd git 
 master on NetBSD, then porting druntime and phobos master.  
 Porting dmd to NetBSD/x86 is likely easy: simply follow what 
 the other BSDs have done, as you did for ldc.


Is it possible to compile dmd with ldc?
There are some additional changes in druntime for dmd comparing 
with ldc. So even working NetBSD dmd compiler is not full 
solution. I think new platforms will have troubles until we have 
separate branches for druntime/phobos in LDC/GDC and patch 
direction master -> LDC/GDC.



 Feb 03 2016



  parent reply Joakim <dlang joakim.fea.st>  writes:


On Wednesday, 3 February 2016 at 10:21:37 UTC, Nikolay wrote:

 On Wednesday, 3 February 2016 at 06:25:49 UTC, Joakim wrote:

 On Tuesday, 2 February 2016 at 18:45:20 UTC, Nikolay wrote:

 You're probably better off porting dmd 2.068 first (as it's 
 the last dmd written wholly in C++), using it to compile dmd 
 git master on NetBSD, then porting druntime and phobos master.
  Porting dmd to NetBSD/x86 is likely easy: simply follow what 
 the other BSDs have done, as you did for ldc.


 Is it possible to compile dmd with ldc?


It should be, as ldc compilation of dmd master is tested 
continuously by one of the auto-testers:

https://github.com/D-Programming-Language/dmd/blob/master/.travis.yml#L5
https://travis-ci.org/D-Programming-Language/dmd/builds/106867288


 There are some additional changes in druntime for dmd comparing 
 with ldc. So even working NetBSD dmd compiler is not full 
 solution. I think new platforms will have troubles until we 
 have separate branches for druntime/phobos in LDC/GDC and patch 
 direction master -> LDC/GDC.


It is possible to do both, ie the changes between the druntime 
used by dmd and ldc are not significant.  Using dmd on NetBSD may 
require some additional changes to its druntime, but likely not 
much.

On Wednesday, 3 February 2016 at 17:36:36 UTC, Nikolay wrote:

 I decided try to repeat LDC team forkflow and cherry-pick my 
 commit to ldc branch (druntime). There are a couple simple 
 conflicts in src/core/stdc/locale.d & 
 src/core/sys/posix/signal.d, and one absent (in master) file: 
 src/rt/sections_ldc.d
 I successfully build and run druntime unittests on NetBSD after 
 resolving conflicts.

 PS
 I created new pull request to druntime: 
 https://github.com/D-Programming-Language/druntime/pull/1492


So you built this druntime with dmd on NetBSD?



 Feb 03 2016



  parent  Nikolay <sibnick gmail.com>  writes:


On Thursday, 4 February 2016 at 05:19:06 UTC, Joakim wrote:


 On Wednesday, 3 February 2016 at 17:36:36 UTC, Nikolay wrote:

 I decided try to repeat LDC team forkflow and cherry-pick my 
 commit to ldc branch (druntime). There are a couple simple 
 conflicts in src/core/stdc/locale.d & 
 src/core/sys/posix/signal.d, and one absent (in master) file: 
 src/rt/sections_ldc.d
 I successfully build and run druntime unittests on NetBSD 
 after resolving conflicts.

 PS
 I created new pull request to druntime: 
 https://github.com/D-Programming-Language/druntime/pull/1492


 So you built this druntime with dmd on NetBSD?


No I use ldc master for building druntime/phobos on NetBSD



 Feb 04 2016



prev sibling  parent  Nikolay <sibnick gmail.com>  writes:


On Wednesday, 3 February 2016 at 06:25:49 UTC, Joakim wrote:

 On Tuesday, 2 February 2016 at 18:45:20 UTC, Nikolay wrote:

 You're probably better off porting dmd 2.068 first (as it's the 
 last dmd written wholly in C++), using it to compile dmd git 
 master on NetBSD, then porting druntime and phobos master.  
 Porting dmd to NetBSD/x86 is likely easy: simply follow what 
 the other BSDs have done, as you did for ldc.


I decided try to repeat LDC team forkflow and cherry-pick my 
commit to ldc branch (druntime). There are a couple simple 
conflicts in src/core/stdc/locale.d & 
src/core/sys/posix/signal.d, and one absent (in master) file: 
src/rt/sections_ldc.d
I successfully build and run druntime unittests on NetBSD after 
resolving conflicts.

PS
I created new pull request to druntime: 
https://github.com/D-Programming-Language/druntime/pull/1492



 Feb 03 2016



prev sibling next sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 2 Feb 2016 7:50 pm, "Nikolay via Digitalmars-d" <
digitalmars-d puremagic.com> wrote:

 I am porting LDC/phobos/druntime to NetBSD. Currently my patch is merged


into LDC master. I have several questions about phobos/druntime and general
workflow.

 As I can understand I should prepare pull requests for phobos/druntime


master branches. LDC team will merge/cherry-pick changes into ldc branch
from master later. Is it correct workflow?  Because it means that I can=E2=
=80=99t
check my patch: there is no dmd compiler for NetBSD + phobos/druntime
master branches.

 Also I have a set of issues with real (long double) type. Several


functions absent in NetBSD for real type (more specific - they are just
aliases for functions with double). Other functions return less accurate
values.

 I have to disable a couple unit tests for NetBSD.
 E.g. std/conv.d:2835 assert(to!string(r) =3D=3D to!string(real.min_normal=


))

 see related question:


http://stackoverflow.com/questions/35090322/netbsd-long-double-trouble

 But also there are a set of unit tests where I have to reduce accuracy.


This problem affects std/internal/math/gammafunction.d, std/math.d,
std/numeric.d, std/complex.d

 E.g.  std/complex.d:792
 assert(sin(complex(2.0L, 0)) =3D=3D std.math.sin(2.0L));
 I replace with:
 assert(feqrel(sin(complex(2.0L, 0)).re, std.math.sin(2.0L)) >


real.mant_dig-10);

 My question is: should I wrap such changes with version(NetBSD) statement


or it is acceptable reduce accuracy for all platforms?

 PS
 You can look to my code here (netbsd_patch branch):
 https://github.com/nrTQgc/druntime
 https://github.com/nrTQgc/phobos


Is NetBSD similar to FreeBSD in that you have 80-bit reals but only the
first 53 bits of the mantissa are used?

I don't know how accurate LDC is when it sets up compile time float
properties.  But what do you get when:

pragma (msg, real.sizeof);
pragma (msg, real.mant_dig);

Alternatively, use a C program with printfs for:

sizeof(long double)
LDBL_MANT_DIG

To make sure both ldc and gcc are agreeable.

Have a look at rt/dmain2.d (look for fldcw) if the problem is because of
what I describe above.



 Feb 02 2016



  parent reply Nikolay <sibnick gmail.com>  writes:


On Wednesday, 3 February 2016 at 07:45:23 UTC, Iain Buclaw wrote:

 On 2 Feb 2016 7:50 pm, "Nikolay via Digitalmars-d" < 
 digitalmars-d puremagic.com> wrote:

 Is NetBSD similar to FreeBSD in that you have 80-bit reals but 
 only the first 53 bits of the mantissa are used?

 I don't know how accurate LDC is when it sets up compile time 
 float properties.  But what do you get when:

 pragma (msg, real.sizeof);
 pragma (msg, real.mant_dig);

 Alternatively, use a C program with printfs for:

 sizeof(long double)
 LDBL_MANT_DIG

 To make sure both ldc and gcc are agreeable.

 Have a look at rt/dmain2.d (look for fldcw) if the problem is 
 because of
 what I describe above.


I suppose it is not similar FreeBSD ( according 
https://www.netbsd.org/changes/changes-7.0.html#x86 )



 Feb 03 2016



  parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 3 February 2016 at 10:25, Nikolay via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Wednesday, 3 February 2016 at 07:45:23 UTC, Iain Buclaw wrote:


 On 2 Feb 2016 7:50 pm, "Nikolay via Digitalmars-d" <
 digitalmars-d puremagic.com> wrote:

 Is NetBSD similar to FreeBSD in that you have 80-bit reals but only the
 first 53 bits of the mantissa are used?

 I don't know how accurate LDC is when it sets up compile time float
 properties.  But what do you get when:

 pragma (msg, real.sizeof);
 pragma (msg, real.mant_dig);

 Alternatively, use a C program with printfs for:

 sizeof(long double)
 LDBL_MANT_DIG

 To make sure both ldc and gcc are agreeable.

 Have a look at rt/dmain2.d (look for fldcw) if the problem is because of
 what I describe above.


 I suppose it is not similar FreeBSD ( according
 https://www.netbsd.org/changes/changes-7.0.html#x86 )


That is a relatively recent change with respect to the age of NetBSD.  :-)

I guess that in LDC the intrinsic for std.math.sin is forwarded to NetBSD
libm.sinl?



 Feb 03 2016



  parent  Nikolay <sibnick gmail.com>  writes:


On Wednesday, 3 February 2016 at 10:36:31 UTC, Iain Buclaw wrote:

 On 3 February 2016 at 10:25, Nikolay via Digitalmars-d < 
 digitalmars-d puremagic.com> wrote:


 That is a relatively recent change with respect to the age of 
 NetBSD.  :-)

 I guess that in LDC the intrinsic for std.math.sin is forwarded 
 to NetBSD libm.sinl?


Yes but https://www.netbsd.org/changes/changes-7.0.html#libm :
Most transcendent functions are stubbed out



 Feb 03 2016



prev sibling  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 2/2/2016 10:45 AM, Nikolay wrote:

 My question is: should I wrap such changes with version(NetBSD) statement


Yes.


 or it is acceptable reduce accuracy for all platforms?


No. There have been many attempts to do this to Phobos, and it wrecks the 
usefulness of the tests as it hides buggy, crappy algorithms.



 Feb 03 2016



  parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 4 Feb 2016 7:25 am, "Walter Bright via Digitalmars-d" <
digitalmars-d puremagic.com> wrote:

 On 2/2/2016 10:45 AM, Nikolay wrote:

 My question is: should I wrap such changes with version(NetBSD) statement



 Yes.



 or it is acceptable reduce accuracy for all platforms?



 No. There have been many attempts to do this to Phobos, and it wrecks the


usefulness of the tests as it hides buggy, crappy algorithms.


Actually, I found the reverse! One of the inline assembly version paths are
correct up to double precision, the platform agnostic path I ported does
not have this problem, but fails the tests.

Adjusting them will *expose* the buggy algorithm for what it really is.



 Feb 03 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 2/3/2016 11:41 PM, Iain Buclaw via Digitalmars-d wrote:

 Actually, I found the reverse! One of the inline assembly version paths are
 correct up to double precision, the platform agnostic path I ported does not
 have this problem, but fails the tests.

 Adjusting them will *expose* the buggy algorithm for what it really is.


I don't understand. How can reducing precision of the test expose bugs?



 Feb 04 2016



  parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 4 February 2016 at 12:29, Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 2/3/2016 11:41 PM, Iain Buclaw via Digitalmars-d wrote:


 Actually, I found the reverse! One of the inline assembly version paths
 are
 correct up to double precision, the platform agnostic path I ported does
 not
 have this problem, but fails the tests.

 Adjusting them will *expose* the buggy algorithm for what it really is.


 I don't understand. How can reducing precision of the test expose bugs?


The literal being used to compare the result was wrong (by 12 mantissa
bits) in the first place.



 Feb 04 2016



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 2/4/2016 4:27 AM, Iain Buclaw via Digitalmars-d wrote:

 The literal being used to compare the result was wrong (by 12 mantissa bits) in
 the first place.



Then the literal should be fixed, not degrade the test.



 Feb 04 2016