www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - questions about NetBSD port

reply Nikolay <sibnick gmail.com> writes:
I am porting LDC/phobos/druntime to NetBSD. Currently my patch is 
merged into LDC master. I have several questions about 
phobos/druntime and general workflow.
As I can understand I should prepare pull requests for 
phobos/druntime master branches. LDC team will merge/cherry-pick 
changes into ldc branch from master later. Is it correct 
workflow?  Because it means that I can’t check my patch: there is 
no dmd compiler for NetBSD + phobos/druntime master branches.

Also I have a set of issues with real (long double) type. Several 
functions absent in NetBSD for real type (more specific - they 
are just aliases for functions with double). Other functions 
return less accurate values.
I have to disable a couple unit tests for NetBSD.
E.g. std/conv.d:2835 assert(to!string(r) == 
to!string(real.min_normal))
see related question:  
http://stackoverflow.com/questions/35090322/netbsd-long-double-trouble
But also there are a set of unit tests where I have to reduce 
accuracy. This problem affects std/internal/math/gammafunction.d, 
std/math.d, std/numeric.d, std/complex.d
E.g.  std/complex.d:792
assert(sin(complex(2.0L, 0)) == std.math.sin(2.0L));
I replace with:
assert(feqrel(sin(complex(2.0L, 0)).re, std.math.sin(2.0L)) > 
real.mant_dig-10);
My question is: should I wrap such changes with version(NetBSD) 
statement or it is acceptable reduce accuracy for all platforms?

PS
You can look to my code here (netbsd_patch branch):
https://github.com/nrTQgc/druntime
https://github.com/nrTQgc/phobos
Feb 02 2016
next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Tuesday, 2 February 2016 at 18:45:20 UTC, Nikolay wrote:
 I am porting LDC/phobos/druntime to NetBSD. Currently my patch 
 is merged into LDC master. I have several questions about 
 phobos/druntime and general workflow.
 As I can understand I should prepare pull requests for 
 phobos/druntime master branches. LDC team will 
 merge/cherry-pick changes into ldc branch from master later. Is 
 it correct workflow?  Because it means that I can’t check my 
 patch: there is no dmd compiler for NetBSD + phobos/druntime 
 master branches.
You're probably better off porting dmd 2.068 first (as it's the last dmd written wholly in C++), using it to compile dmd git master on NetBSD, then porting druntime and phobos master. Porting dmd to NetBSD/x86 is likely easy: simply follow what the other BSDs have done, as you did for ldc.
 Also I have a set of issues with real (long double) type. 
 Several functions absent in NetBSD for real type (more specific 
 - they are just aliases for functions with double). Other 
 functions return less accurate values.
 I have to disable a couple unit tests for NetBSD.
 E.g. std/conv.d:2835 assert(to!string(r) == 
 to!string(real.min_normal))
 see related question:  
 http://stackoverflow.com/questions/35090322/netbsd-long-double-trouble
 But also there are a set of unit tests where I have to reduce 
 accuracy. This problem affects 
 std/internal/math/gammafunction.d, std/math.d, std/numeric.d, 
 std/complex.d
 E.g.  std/complex.d:792
 assert(sin(complex(2.0L, 0)) == std.math.sin(2.0L));
 I replace with:
 assert(feqrel(sin(complex(2.0L, 0)).re, std.math.sin(2.0L)) > 
 real.mant_dig-10);
 My question is: should I wrap such changes with version(NetBSD) 
 statement or it is acceptable reduce accuracy for all platforms?
This is the same issue Win64 and Android have had. You should already see version blocks for those in some places, you can do the same. Take a look at my Android patches for similar tweaks that are still unmerged: https://gist.github.com/joakim-noah/d936d6a339426ad1fac3 https://gist.github.com/joakim-noah/5c03801fa6c59b1e90df
Feb 02 2016
next sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 3 Feb 2016 7:30 am, "Joakim via Digitalmars-d" <
digitalmars-d puremagic.com> wrote:
 On Tuesday, 2 February 2016 at 18:45:20 UTC, Nikolay wrote:
 I am porting LDC/phobos/druntime to NetBSD. Currently my patch is merged
into LDC master. I have several questions about phobos/druntime and general workflow.
 As I can understand I should prepare pull requests for phobos/druntime
master branches. LDC team will merge/cherry-pick changes into ldc branch from master later. Is it correct workflow? Because it means that I can=E2= =80=99t check my patch: there is no dmd compiler for NetBSD + phobos/druntime master branches.
 You're probably better off porting dmd 2.068 first (as it's the last dmd
written wholly in C++), using it to compile dmd git master on NetBSD, then porting druntime and phobos master. Porting dmd to NetBSD/x86 is likely easy: simply follow what the other BSDs have done, as you did for ldc.

I doubt it. Using ldc/gdc usually means that it's just the library you need
to play with, not the compiler.
Feb 02 2016
parent reply Joakim <dlang joakim.fea.st> writes:
On Wednesday, 3 February 2016 at 07:01:10 UTC, Iain Buclaw wrote:
 On 3 Feb 2016 7:30 am, "Joakim via Digitalmars-d" < 
 digitalmars-d puremagic.com> wrote:
 On Tuesday, 2 February 2016 at 18:45:20 UTC, Nikolay wrote:
 I am porting LDC/phobos/druntime to NetBSD. Currently my 
 patch is merged
into LDC master. I have several questions about phobos/druntime and general workflow.
 As I can understand I should prepare pull requests for 
 phobos/druntime
master branches. LDC team will merge/cherry-pick changes into ldc branch from master later. Is it correct workflow? Because it means that I can’t check my patch: there is no dmd compiler for NetBSD + phobos/druntime master branches.
 You're probably better off porting dmd 2.068 first (as it's 
 the last dmd
written wholly in C++), using it to compile dmd git master on NetBSD, then porting druntime and phobos master. Porting dmd to NetBSD/x86 is likely easy: simply follow what the other BSDs have done, as you did for ldc.

 I doubt it. Using ldc/gdc usually means that it's just the 
 library you need to play with, not the compiler.
The problem is that ldc/gdc are not on 2.069, let alone the just-released 2.070. So as he said, if he really wants to test his druntime/phobos PRs against master _on_ NetBSD, he has no choice but to port dmd. I haven't had to do so myself, but looking at how it's been done for other BSDs, it looks fairly easy to me.
Feb 02 2016
parent Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 3 Feb 2016 8:35 am, "Joakim via Digitalmars-d" <
digitalmars-d puremagic.com> wrote:
 On Wednesday, 3 February 2016 at 07:01:10 UTC, Iain Buclaw wrote:
 On 3 Feb 2016 7:30 am, "Joakim via Digitalmars-d" <
digitalmars-d puremagic.com> wrote:
 On Tuesday, 2 February 2016 at 18:45:20 UTC, Nikolay wrote:
 I am porting LDC/phobos/druntime to NetBSD. Currently my patch is
merged
 into LDC master. I have several questions about phobos/druntime and
general workflow.
 As I can understand I should prepare pull requests for phobos/druntime
master branches. LDC team will merge/cherry-pick changes into ldc branch
from master later. Is it correct workflow? Because it means that I can=E2= =80=99t check my patch: there is no dmd compiler for NetBSD + phobos/druntime master branches.
 You're probably better off porting dmd 2.068 first (as it's the last dm=
d
 written wholly in C++), using it to compile dmd git master on NetBSD,
then porting druntime and phobos master. Porting dmd to NetBSD/x86 is likely easy: simply follow what the other BSDs have done, as you did for ldc.

 I doubt it. Using ldc/gdc usually means that it's just the library you
need to play with, not the compiler.
 The problem is that ldc/gdc are not on 2.069, let alone the just-released
2.070. So as he said, if he really wants to test his druntime/phobos PRs against master _on_ NetBSD, he has no choice but to port dmd. I haven't had to do so myself, but looking at how it's been done for other BSDs, it looks fairly easy to me. Druntime doesn't change all that much, so it's trivial to rebase to master. As for phobos, the only parts that matter tend to be std.math and co. Maybe there are some places like std.file too, but I've likely been spoiled by using version(GNU) in places where DMD chooses to have a version branch for each platform. :-)
Feb 02 2016
prev sibling next sibling parent reply Nikolay <sibnick gmail.com> writes:
On Wednesday, 3 February 2016 at 06:25:49 UTC, Joakim wrote:
 On Tuesday, 2 February 2016 at 18:45:20 UTC, Nikolay wrote:

 You're probably better off porting dmd 2.068 first (as it's the 
 last dmd written wholly in C++), using it to compile dmd git 
 master on NetBSD, then porting druntime and phobos master.  
 Porting dmd to NetBSD/x86 is likely easy: simply follow what 
 the other BSDs have done, as you did for ldc.
Is it possible to compile dmd with ldc? There are some additional changes in druntime for dmd comparing with ldc. So even working NetBSD dmd compiler is not full solution. I think new platforms will have troubles until we have separate branches for druntime/phobos in LDC/GDC and patch direction master -> LDC/GDC.
Feb 03 2016
parent reply Joakim <dlang joakim.fea.st> writes:
On Wednesday, 3 February 2016 at 10:21:37 UTC, Nikolay wrote:
 On Wednesday, 3 February 2016 at 06:25:49 UTC, Joakim wrote:
 On Tuesday, 2 February 2016 at 18:45:20 UTC, Nikolay wrote:

 You're probably better off porting dmd 2.068 first (as it's 
 the last dmd written wholly in C++), using it to compile dmd 
 git master on NetBSD, then porting druntime and phobos master.
  Porting dmd to NetBSD/x86 is likely easy: simply follow what 
 the other BSDs have done, as you did for ldc.
Is it possible to compile dmd with ldc?
It should be, as ldc compilation of dmd master is tested continuously by one of the auto-testers: https://github.com/D-Programming-Language/dmd/blob/master/.travis.yml#L5 https://travis-ci.org/D-Programming-Language/dmd/builds/106867288
 There are some additional changes in druntime for dmd comparing 
 with ldc. So even working NetBSD dmd compiler is not full 
 solution. I think new platforms will have troubles until we 
 have separate branches for druntime/phobos in LDC/GDC and patch 
 direction master -> LDC/GDC.
It is possible to do both, ie the changes between the druntime used by dmd and ldc are not significant. Using dmd on NetBSD may require some additional changes to its druntime, but likely not much. On Wednesday, 3 February 2016 at 17:36:36 UTC, Nikolay wrote:
 I decided try to repeat LDC team forkflow and cherry-pick my 
 commit to ldc branch (druntime). There are a couple simple 
 conflicts in src/core/stdc/locale.d & 
 src/core/sys/posix/signal.d, and one absent (in master) file: 
 src/rt/sections_ldc.d
 I successfully build and run druntime unittests on NetBSD after 
 resolving conflicts.

 PS
 I created new pull request to druntime: 
 https://github.com/D-Programming-Language/druntime/pull/1492
So you built this druntime with dmd on NetBSD?
Feb 03 2016
parent Nikolay <sibnick gmail.com> writes:
On Thursday, 4 February 2016 at 05:19:06 UTC, Joakim wrote:

 On Wednesday, 3 February 2016 at 17:36:36 UTC, Nikolay wrote:
 I decided try to repeat LDC team forkflow and cherry-pick my 
 commit to ldc branch (druntime). There are a couple simple 
 conflicts in src/core/stdc/locale.d & 
 src/core/sys/posix/signal.d, and one absent (in master) file: 
 src/rt/sections_ldc.d
 I successfully build and run druntime unittests on NetBSD 
 after resolving conflicts.

 PS
 I created new pull request to druntime: 
 https://github.com/D-Programming-Language/druntime/pull/1492
So you built this druntime with dmd on NetBSD?
No I use ldc master for building druntime/phobos on NetBSD
Feb 04 2016
prev sibling parent Nikolay <sibnick gmail.com> writes:
On Wednesday, 3 February 2016 at 06:25:49 UTC, Joakim wrote:
 On Tuesday, 2 February 2016 at 18:45:20 UTC, Nikolay wrote:

 You're probably better off porting dmd 2.068 first (as it's the 
 last dmd written wholly in C++), using it to compile dmd git 
 master on NetBSD, then porting druntime and phobos master.  
 Porting dmd to NetBSD/x86 is likely easy: simply follow what 
 the other BSDs have done, as you did for ldc.
I decided try to repeat LDC team forkflow and cherry-pick my commit to ldc branch (druntime). There are a couple simple conflicts in src/core/stdc/locale.d & src/core/sys/posix/signal.d, and one absent (in master) file: src/rt/sections_ldc.d I successfully build and run druntime unittests on NetBSD after resolving conflicts. PS I created new pull request to druntime: https://github.com/D-Programming-Language/druntime/pull/1492
Feb 03 2016
prev sibling next sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 2 Feb 2016 7:50 pm, "Nikolay via Digitalmars-d" <
digitalmars-d puremagic.com> wrote:
 I am porting LDC/phobos/druntime to NetBSD. Currently my patch is merged
into LDC master. I have several questions about phobos/druntime and general workflow.
 As I can understand I should prepare pull requests for phobos/druntime
master branches. LDC team will merge/cherry-pick changes into ldc branch from master later. Is it correct workflow? Because it means that I can=E2= =80=99t check my patch: there is no dmd compiler for NetBSD + phobos/druntime master branches.
 Also I have a set of issues with real (long double) type. Several
functions absent in NetBSD for real type (more specific - they are just aliases for functions with double). Other functions return less accurate values.
 I have to disable a couple unit tests for NetBSD.
 E.g. std/conv.d:2835 assert(to!string(r) =3D=3D to!string(real.min_normal=
))
 see related question:
http://stackoverflow.com/questions/35090322/netbsd-long-double-trouble
 But also there are a set of unit tests where I have to reduce accuracy.
This problem affects std/internal/math/gammafunction.d, std/math.d, std/numeric.d, std/complex.d
 E.g.  std/complex.d:792
 assert(sin(complex(2.0L, 0)) =3D=3D std.math.sin(2.0L));
 I replace with:
 assert(feqrel(sin(complex(2.0L, 0)).re, std.math.sin(2.0L)) >
real.mant_dig-10);
 My question is: should I wrap such changes with version(NetBSD) statement
or it is acceptable reduce accuracy for all platforms?
 PS
 You can look to my code here (netbsd_patch branch):
 https://github.com/nrTQgc/druntime
 https://github.com/nrTQgc/phobos
Is NetBSD similar to FreeBSD in that you have 80-bit reals but only the first 53 bits of the mantissa are used? I don't know how accurate LDC is when it sets up compile time float properties. But what do you get when: pragma (msg, real.sizeof); pragma (msg, real.mant_dig); Alternatively, use a C program with printfs for: sizeof(long double) LDBL_MANT_DIG To make sure both ldc and gcc are agreeable. Have a look at rt/dmain2.d (look for fldcw) if the problem is because of what I describe above.
Feb 02 2016
parent reply Nikolay <sibnick gmail.com> writes:
On Wednesday, 3 February 2016 at 07:45:23 UTC, Iain Buclaw wrote:
 On 2 Feb 2016 7:50 pm, "Nikolay via Digitalmars-d" < 
 digitalmars-d puremagic.com> wrote:

 Is NetBSD similar to FreeBSD in that you have 80-bit reals but 
 only the first 53 bits of the mantissa are used?

 I don't know how accurate LDC is when it sets up compile time 
 float properties.  But what do you get when:

 pragma (msg, real.sizeof);
 pragma (msg, real.mant_dig);

 Alternatively, use a C program with printfs for:

 sizeof(long double)
 LDBL_MANT_DIG

 To make sure both ldc and gcc are agreeable.

 Have a look at rt/dmain2.d (look for fldcw) if the problem is 
 because of
 what I describe above.
I suppose it is not similar FreeBSD ( according https://www.netbsd.org/changes/changes-7.0.html#x86 )
Feb 03 2016
parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 3 February 2016 at 10:25, Nikolay via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Wednesday, 3 February 2016 at 07:45:23 UTC, Iain Buclaw wrote:

 On 2 Feb 2016 7:50 pm, "Nikolay via Digitalmars-d" <
 digitalmars-d puremagic.com> wrote:

 Is NetBSD similar to FreeBSD in that you have 80-bit reals but only the
 first 53 bits of the mantissa are used?

 I don't know how accurate LDC is when it sets up compile time float
 properties.  But what do you get when:

 pragma (msg, real.sizeof);
 pragma (msg, real.mant_dig);

 Alternatively, use a C program with printfs for:

 sizeof(long double)
 LDBL_MANT_DIG

 To make sure both ldc and gcc are agreeable.

 Have a look at rt/dmain2.d (look for fldcw) if the problem is because of
 what I describe above.
I suppose it is not similar FreeBSD ( according https://www.netbsd.org/changes/changes-7.0.html#x86 )
That is a relatively recent change with respect to the age of NetBSD. :-) I guess that in LDC the intrinsic for std.math.sin is forwarded to NetBSD libm.sinl?
Feb 03 2016
parent Nikolay <sibnick gmail.com> writes:
On Wednesday, 3 February 2016 at 10:36:31 UTC, Iain Buclaw wrote:
 On 3 February 2016 at 10:25, Nikolay via Digitalmars-d < 
 digitalmars-d puremagic.com> wrote:


 That is a relatively recent change with respect to the age of 
 NetBSD.  :-)

 I guess that in LDC the intrinsic for std.math.sin is forwarded 
 to NetBSD libm.sinl?
Yes but https://www.netbsd.org/changes/changes-7.0.html#libm : Most transcendent functions are stubbed out
Feb 03 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/2/2016 10:45 AM, Nikolay wrote:
 My question is: should I wrap such changes with version(NetBSD) statement
Yes.
 or it is acceptable reduce accuracy for all platforms?
No. There have been many attempts to do this to Phobos, and it wrecks the usefulness of the tests as it hides buggy, crappy algorithms.
Feb 03 2016
parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 4 Feb 2016 7:25 am, "Walter Bright via Digitalmars-d" <
digitalmars-d puremagic.com> wrote:
 On 2/2/2016 10:45 AM, Nikolay wrote:
 My question is: should I wrap such changes with version(NetBSD) statement
Yes.
 or it is acceptable reduce accuracy for all platforms?
No. There have been many attempts to do this to Phobos, and it wrecks the
usefulness of the tests as it hides buggy, crappy algorithms.

Actually, I found the reverse! One of the inline assembly version paths are
correct up to double precision, the platform agnostic path I ported does
not have this problem, but fails the tests.

Adjusting them will *expose* the buggy algorithm for what it really is.
Feb 03 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/3/2016 11:41 PM, Iain Buclaw via Digitalmars-d wrote:
 Actually, I found the reverse! One of the inline assembly version paths are
 correct up to double precision, the platform agnostic path I ported does not
 have this problem, but fails the tests.

 Adjusting them will *expose* the buggy algorithm for what it really is.
I don't understand. How can reducing precision of the test expose bugs?
Feb 04 2016
parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 4 February 2016 at 12:29, Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 2/3/2016 11:41 PM, Iain Buclaw via Digitalmars-d wrote:

 Actually, I found the reverse! One of the inline assembly version paths
 are
 correct up to double precision, the platform agnostic path I ported does
 not
 have this problem, but fails the tests.

 Adjusting them will *expose* the buggy algorithm for what it really is.
I don't understand. How can reducing precision of the test expose bugs?
The literal being used to compare the result was wrong (by 12 mantissa bits) in the first place.
Feb 04 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/4/2016 4:27 AM, Iain Buclaw via Digitalmars-d wrote:
 The literal being used to compare the result was wrong (by 12 mantissa bits) in
 the first place.
Then the literal should be fixed, not degrade the test.
Feb 04 2016