www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.ldc - Disabling http tests for guix

reply fredmanglis <fredmanglis gmail.com> writes:
Hi.

I'm packaging ldc-1.1.0-beta6 for guix, which does not allow for 
network access when building the package, leading to errors, and 
timeouts when the tests are run.

Is there a way to deactivate such tests, say a flag passed to the 
build system or such? I would be grateful if you'd point me to 
one.

In the meantime, I have had to make use of huge patches to 
deactivate the tests to make the builds pass, which does not seem 
sustainable
Jan 04 2017
next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Wednesday, 4 January 2017 at 18:40:57 UTC, fredmanglis wrote:

 Hi.

 I'm packaging ldc-1.1.0-beta6 for guix, which does not allow 
 for network access when building the package, leading to 
 errors, and timeouts when the tests are run.

 Is there a way to deactivate such tests, say a flag passed to 
 the build system or such? I would be grateful if you'd point me 
 to one.

 In the meantime, I have had to make use of huge patches to 
 deactivate the tests to make the builds pass, which does not 
 seem sustainable


Why are you running the tests at all for building a package?

The command

make ldmd2 druntime-ldc phobos2-ldc

will just build the compiler and standard library, along with the 
dmd-compatible driver. I assume you are running make alone, which 
might run the tests too.  Add druntime-ldc-debug and 
phobos2-ldc-debug to the list to get everything that normally 
comes with an ldc release.
Jan 04 2017
parent reply fredmanglis <fredmanglis gmail.com> writes:
On Wednesday, 4 January 2017 at 19:15:09 UTC, Joakim wrote:

 Why are you running the tests at all for building a package?


Well, as a principle, the Guix package manager runs tests while 
building the package as part of ensuring that the package was 
built correctly. To ensure reproducibility of the builds, 
however, the environment set up cuts away anything that 
introduces variations, network access being one of the major ones.

As such, I need that the tests are run, otherwise the package 
definition will not be accepted (I can't simply cop-out and 
disable all the tests).

To be clear, the building phase completes successfully, but the 
testing phase fails, and Guix requires that all the stages pass 
successfully.
Jan 04 2017
parent reply Joakim <dlang joakim.fea.st> writes:
On Thursday, 5 January 2017 at 06:18:25 UTC, fredmanglis wrote:

 On Wednesday, 4 January 2017 at 19:15:09 UTC, Joakim wrote:

 Why are you running the tests at all for building a package?


 Well, as a principle, the Guix package manager runs tests while 
 building the package as part of ensuring that the package was 
 built correctly. To ensure reproducibility of the builds, 
 however, the environment set up cuts away anything that 
 introduces variations, network access being one of the major 
 ones.

 As such, I need that the tests are run, otherwise the package 
 definition will not be accepted (I can't simply cop-out and 
 disable all the tests).

 To be clear, the building phase completes successfully, but the 
 testing phase fails, and Guix requires that all the stages pass 
 successfully.


I see, do you run all the ldc tests, including the dmd testsuite? 
Can you post the patch you're using now, so we can get an idea of 
what you had to disable?  I haven't had many problems running the 
tests even without network access, I believe they should just 
keep going.

This could be pulled upstream, especially since they seem to be 
focusing on getting into distros recently:

http://forum.dlang.org/thread/o3cdl9$ecs$1 digitalmars.com

Maybe we can get a PR in upstream to selectively disable tests 
that error out in your test environment, which other packagers 
might want too and might save you the trouble of maintaining a 
patch yourself.  Let me know and I can help you with that.
Jan 04 2017
parent reply fredmanglis <fredmanglis gmail.com> writes:
On Thursday, 5 January 2017 at 07:06:11 UTC, Joakim wrote:

 I see, do you run all the ldc tests, including the dmd 
 testsuite? Can you post the patch you're using now, so we can 
 get an idea of what you had to disable?  I haven't had many 
 problems running the tests even without network access, I 
 believe they should just keep going.


Yes. The system runs all the tests, including dmd-testsuite.

As for the patches, there are 2, one for phobos, and one for dmd. 
I do not see an attach file option, and I don't know if it is 
acceptable to just dump the patches here: what's a convenient way 
to send the patches?
Jan 05 2017
parent reply David Nadlinger via digitalmars-d-ldc <digitalmars-d-ldc puremagic.com> writes:
On Thu, Jan 5, 2017 at 5:17 PM, fredmanglis via digitalmars-d-ldc
<digitalmars-d-ldc puremagic.com> wrote:

 As for the patches, there are 2, one for phobos, and one for dmd. I do not
 see an attach file option, and I don't know if it is acceptable to just dump
 the patches here: what's a convenient way to send the patches?


The way to submit them for inclusion would be as a PR at
https://github.com/dlang/dmd and https://github.com/dlang/phobos.

For just sharing them here, you can either attach them when using the
email interface (they'll also show up in the web ui), or just paste
them somewhere (Gist, ...) and post the link here.

IIRC there is already something like PHOBOS_TEST_ALLOW_NET for Phobos,
though? (Not on my dev machine right now.)

 — David
Jan 05 2017
parent reply fredmanglis <fredmanglis gmail.com> writes:
On Thursday, 5 January 2017 at 16:43:47 UTC, David Nadlinger 
wrote:

 On Thu, Jan 5, 2017 at 5:17 PM, fredmanglis via 
 digitalmars-d-ldc <digitalmars-d-ldc puremagic.com> wrote:

 As for the patches, there are 2, one for phobos, and one for 
 dmd. I do not see an attach file option, and I don't know if 
 it is acceptable to just dump the patches here: what's a 
 convenient way to send the patches?


 The way to submit them for inclusion would be as a PR at 
 https://github.com/dlang/dmd and 
 https://github.com/dlang/phobos.


For now, I am not submitting the patches for inclusion, since as 
the thread notes, they are a shim, to get Guix to pass the tests


 For just sharing them here, you can either attach them when 
 using the
 email interface (they'll also show up in the web ui), or just 
 paste
 them somewhere (Gist, ...) and post the link here.


The patch I made for phobos: 
https://gist.github.com/fredmanglis/02d5255feba18167aaa6b23942553928#file-ldc1-1-0-disable-phobos-tests-patch

The patch I made for dmd-testsuite: 
https://gist.github.com/fredmanglis/a423c18abad0bfd3b27be49aa722dcc5#file-ldc1-1-0-disable-dmd-tests-patch


 IIRC there is already something like PHOBOS_TEST_ALLOW_NET for 
 Phobos, though? (Not on my dev machine right now.)


Okay. That seems promising. Let me look for that
Jan 05 2017
parent reply Joakim <dlang joakim.fea.st> writes:
On Thursday, 5 January 2017 at 17:31:38 UTC, fredmanglis wrote:

 For just sharing them here, you can either attach them when 
 using the
 email interface (they'll also show up in the web ui), or just 
 paste
 them somewhere (Gist, ...) and post the link here.


 The patch I made for phobos: 
 https://gist.github.com/fredmanglis/02d5255feba18167aaa6b23942553928#file-ldc1-1-0-disable-phobos-tests-patch

 The patch I made for dmd-testsuite: 
 https://gist.github.com/fredmanglis/a423c18abad0bfd3b27be49aa722dcc5#file-ldc1-1-0-disable-dmd-tests-patch


Seem fairly small, but I could see why you wouldn't want to 
maintain them yourself.  If you're up for it, we could probably 
add a version flag to the test suite to disable these sections 
for packagers.


 IIRC there is already something like PHOBOS_TEST_ALLOW_NET for 
 Phobos, though? (Not on my dev machine right now.)


 Okay. That seems promising. Let me look for that


Never heard of it, may have been removed.
Jan 05 2017
parent reply fredmanglis <fredmanglis gmail.com> writes:
On Thursday, 5 January 2017 at 23:18:36 UTC, Joakim wrote:

 Seem fairly small, but I could see why you wouldn't want to 
 maintain them yourself.  If you're up for it, we could probably 
 add a version flag to the test suite to disable these sections 
 for packagers.


I am up for adding the version flag, though I might have to learn 
D first.


 IIRC there is already something like PHOBOS_TEST_ALLOW_NET 
 for Phobos, though? (Not on my dev machine right now.)


 Okay. That seems promising. Let me look for that


 Never heard of it, may have been removed.


Yeah, My searches for this do not seem to be conclusive yet. I 
will look at it some more though to ensure I am not missing 
anything.
Jan 06 2017
parent reply Pjotr Prins <pjotr.public12 thebird.nl> writes:
On Friday, 6 January 2017 at 13:39:31 UTC, fredmanglis wrote:

 On Thursday, 5 January 2017 at 23:18:36 UTC, Joakim wrote:

 Seem fairly small, but I could see why you wouldn't want to 
 maintain them yourself.  If you're up for it, we could 
 probably add a version flag to the test suite to disable these 
 sections for packagers.




Similar to StdUnittest in 
https://github.com/dlang/phobos/pull/5927, I propose we
add version(StdNetworkunittest) and/or version(StdEnvUnittest) 
for those tests that require internet access or access to a 
directory outside the source tree (e.g. $TMPDIR) as these may not 
be available in containers and other restrictive build systems. 
Does that make sense?
Feb 11 2018
parent reply Seb <seb wilzba.ch> writes:
On Sunday, 11 February 2018 at 15:39:18 UTC, Pjotr Prins wrote:

 On Friday, 6 January 2017 at 13:39:31 UTC, fredmanglis wrote:

 On Thursday, 5 January 2017 at 23:18:36 UTC, Joakim wrote:

 Seem fairly small, but I could see why you wouldn't want to 
 maintain them yourself.  If you're up for it, we could 
 probably add a version flag to the test suite to disable 
 these sections for packagers.




 Similar to StdUnittest in 
 https://github.com/dlang/phobos/pull/5927, I propose we
 add version(StdNetworkunittest) and/or version(StdEnvUnittest) 
 for those tests that require internet access or access to a 
 directory outside the source tree (e.g. $TMPDIR) as these may 
 not be available in containers and other restrictive build 
 systems. Does that make sense?


How about the reverse, s.t. not all build scripts need to be 
updated?
Feb 11 2018
next sibling parent Pjotr Prins <pjotr.public12 thebird.nl> writes:
On Sunday, 11 February 2018 at 19:40:56 UTC, Seb wrote:

 How about the reverse, s.t. not all build scripts need to be 
 updated?


The following would only be for tests that access the internet:

    version(StdEnvUnittest) {
        do something with internet enabled
    }

Not that many tests.
Feb 11 2018
prev sibling parent Pjotr Prins <pjotr.public12 thebird.nl> writes:
On Sunday, 11 February 2018 at 19:40:56 UTC, Seb wrote:

 How about the reverse, s.t. not all build scripts need to be 
 updated?


Can we do something like

!version(ConstrainedEnvUnittest) {
    networking....
}

i.e., a negative? Or do we need to write

version(ConstrainedEnvUnittest) {
    // empty
} else {
    networking...
}

everywhere.

Maybe version is not the best way to do this.
Feb 11 2018
prev sibling parent reply Thomas Mader <thomas.mader gmail.com> writes:
On Wednesday, 4 January 2017 at 18:40:57 UTC, fredmanglis wrote:

 Hi.

 I'm packaging ldc-1.1.0-beta6 for guix, which does not allow 
 for network access when building the package, leading to 
 errors, and timeouts when the tests are run.

 Is there a way to deactivate such tests, say a flag passed to 
 the build system or such? I would be grateful if you'd point me 
 to one.

 In the meantime, I have had to make use of huge patches to 
 deactivate the tests to make the builds pass, which does not 
 seem sustainable


I am packaging ldc for NixOS and I was having the same problem 
there.
I solved it by putting the tests with special needs into a fixed 
output derivation.
That's how it is called in Nix world. Fixed output derivation's 
always produce the same output and are therefore allowed to 
access the network. They are also used to implement all kinds of 
fetchers to get the source archives from GitHub and the like.
guix sure has something equal because it basically is the same 
system. They even use parts of Nix.

Here you can take a look into the current package: 
https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/compilers/ldc/default.nix
Feb 12 2018
parent reply Pjotr Prins <pjotr.public12 thebird.nl> writes:
On Monday, 12 February 2018 at 21:19:35 UTC, Thomas Mader wrote:

 I am packaging ldc for NixOS and I was having the same problem 
 there. I solved it by putting the tests with special needs into 
 a fixed output derivation.
 That's how it is called in Nix world. Fixed output derivation's 
 always produce the same output and are therefore allowed to 
 access the network. They are also used to implement all kinds 
 of fetchers to get the source archives from GitHub and the like.


That is an interesting approach and I will discuss it with 
others. So an output derivation is an output that gets generated 
and allows for running it after the installation phase - giving 
it an opportunity to run tests. It is an interesting approach and 
works because the store is 'immutable'.

Two downsides: (1) is you can not assume there is a network on 
the build machine. We need to be able to build and test packages 
in isolation on restricted computers to make sure no one can 
tamper. (2) it allows pulling software, as you state, which 
changes the state of test system and makes it potentially 
non-reproducible. I don't think the Guix folks thing that this is 
the way forward. Impurity is a big no.


 guix sure has something equal because it basically is the same 
 system. They even use parts of Nix.


Currently Nix and Guix share the build daemon. Otherwise Nix and 
Guix are different animals altogether. I moved from Nix to Guix a 
few years back and know both systems intimately.


 Here you can take a look into the current package: 
 https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/compilers/ldc/default.nix


here is mine (almost done, I need to remove some lines):

     
https://gitlab.com/genenetwork/guix/blob/dlang/gnu/packages/ldc.scm#L183

to me the difference is that (if you think away the LISP braces) 
Guix looks a lot more readable and the logic is clear from the 
expression (some of it is probably a matter of taste). I also 
note that the Nix package does not to build shared phobos libs 
and is not running all ldc tests.

But, admittedly, both package descriptions are short and clear.
Feb 15 2018
next sibling parent reply Kagamin <spam here.lot> writes:
On Friday, 16 February 2018 at 06:13:05 UTC, Pjotr Prins wrote:

 Two downsides: (1) is you can not assume there is a network on 
 the build machine. We need to be able to build and test 
 packages in isolation on restricted computers to make sure no 
 one can tamper. (2) it allows pulling software, as you state, 
 which changes the state of test system and makes it potentially 
 non-reproducible. I don't think the Guix folks thing that this 
 is the way forward. Impurity is a big no.


Do you mean phobos curl tests? They use a self-contained test 
server, should be pure.
Feb 16 2018
parent reply Pjotr Prins <pjotr.public12 thebird.nl> writes:
On Friday, 16 February 2018 at 12:00:18 UTC, Kagamin wrote:

 On Friday, 16 February 2018 at 06:13:05 UTC, Pjotr Prins wrote:

 Two downsides: (1) is you can not assume there is a network on 
 the build machine. We need to be able to build and test 
 packages in isolation on restricted computers to make sure no 
 one can tamper. (2) it allows pulling software, as you state, 
 which changes the state of test system and makes it 
 potentially non-reproducible. I don't think the Guix folks 
 thing that this is the way forward. Impurity is a big no.


 Do you mean phobos curl tests? They use a self-contained test 
 server, should be pure.


Those are fine. Current patches to make the build work are

   
https://gitlab.com/genenetwork/guix/blob/dlang/gnu/packages/patches/ldc-1.7.0-disable-phobos-tests.patch

Not too many!

Remaining problems are with assumptions about socket and root 
path. I'll submit some patches if I can make it work. The root 
path can probably be avoided by faking it into the build dir.
Feb 16 2018
parent Thomas Mader <thomas.mader gmail.com> writes:
On Friday, 16 February 2018 at 13:14:22 UTC, Pjotr Prins wrote:

 Those are fine. Current patches to make the build work are

   
 https://gitlab.com/genenetwork/guix/blob/dlang/gnu/packages/patches/ldc-1.7.0-disable-phobos-tests.patch

 Not too many!

 Remaining problems are with assumptions about socket and root 
 path. I'll submit some patches if I can make it work. The root 
 path can probably be avoided by faking it into the build dir.


Solutions to phobos problems already went upstream.
Some of them should be getting out with the next major version 
release for dmd.
You could go through the package script for dmd, some of them 
have comments.

The ones I remember which are fixed: root/homedirectory, datetime 
problems with the path to the timezone db, curl path problems.

The remaining problems I am having seem to be Nix related.
So you shouldn't have other problems with guix than me with Nix. 
It might be a good idea if we follow each other on Github or 
something to prevent double work.
Feb 17 2018
prev sibling next sibling parent Thomas Mader <thomas.mader gmail.com> writes:
On Friday, 16 February 2018 at 06:13:05 UTC, Pjotr Prins wrote:

 That is an interesting approach and I will discuss it with 
 others. So an output derivation is an output that gets 
 generated and allows for running it after the installation 
 phase - giving it an opportunity to run tests. It is an 
 interesting approach and works because the store is 'immutable'.

 Two downsides: (1) is you can not assume there is a network on 
 the build machine. We need to be able to build and test 
 packages in isolation on restricted computers to make sure no 
 one can tamper.


As Kagamin already noted, Phobos isn't running any network tests 
but dub is in it's tests.
But I don't see that as a problem. You always depend on the 
network for all package builds because you need to get the source 
from somewhere.
If you want to be able to build without Internet access you need 
to have it cached somewhere. Either on your machine or on a 
proxy. That's how it works. Is that different with guix?


 (2) it allows pulling software, as you state, which changes the 
 state of test system and makes it potentially non-reproducible. 
 I don't think the Guix folks thing that this is the way 
 forward. Impurity is a big no.


As you already stated a fixed output derivation isn't changing 
the derivation which uses it because the output is the same all 
the time. What happens inside the derivation is of no concern to 
the using derivation as long as it runs through.


 guix sure has something equal because it basically is the same 
 system. They even use parts of Nix.


 Currently Nix and Guix share the build daemon. Otherwise Nix 
 and Guix are different animals altogether. I moved from Nix to 
 Guix a few years back and know both systems intimately.


I decided to go with Nix because the community seems bigger and 
it was the project with the initial idea for such a system.
I don't see me moving but would still be interested in your 
reasons.


 here is mine (almost done, I need to remove some lines):

     
 https://gitlab.com/genenetwork/guix/blob/dlang/gnu/packages/ldc.scm#L183

 to me the difference is that (if you think away the LISP 
 braces) Guix looks a lot more readable and the logic is clear 
 from the expression (some of it is probably a matter of taste). 
 I also note that the Nix package does not to build shared 
 phobos libs and is not running all ldc tests.

 But, admittedly, both package descriptions are short and clear.


Sorry I can't see that. :-)
I knew that guix comes with it's own language and if I remember 
correct I read somewhere that the Nix project lead is not against 
something like that for Nix too but I am not sure if it is 
currently an active topic. But I don't think it is.
I don't care, for me it would be more important to have such a 
system for Windows too to be able to have one package manager for 
the major three OSes.

Does guix come with a nice command line interface? Nix used to 
have multiple commands but with Nix 2.0 all is unified via 'nix 
<subcommand>'.
Feb 17 2018
prev sibling parent Thomas Mader <thomas.mader gmail.com> writes:
On Friday, 16 February 2018 at 06:13:05 UTC, Pjotr Prins wrote:

 I also note that the Nix package does not to build shared 
 phobos libs and is not running all ldc tests.


Which ones are missing?

Why are the shared libs not built?
Feb 17 2018