www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.ldc - Disabling http tests for guix

reply fredmanglis <fredmanglis gmail.com> writes:
Hi.

I'm packaging ldc-1.1.0-beta6 for guix, which does not allow for 
network access when building the package, leading to errors, and 
timeouts when the tests are run.

Is there a way to deactivate such tests, say a flag passed to the 
build system or such? I would be grateful if you'd point me to 
one.

In the meantime, I have had to make use of huge patches to 
deactivate the tests to make the builds pass, which does not seem 
sustainable
Jan 04 2017
next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Wednesday, 4 January 2017 at 18:40:57 UTC, fredmanglis wrote:
 Hi.

 I'm packaging ldc-1.1.0-beta6 for guix, which does not allow 
 for network access when building the package, leading to 
 errors, and timeouts when the tests are run.

 Is there a way to deactivate such tests, say a flag passed to 
 the build system or such? I would be grateful if you'd point me 
 to one.

 In the meantime, I have had to make use of huge patches to 
 deactivate the tests to make the builds pass, which does not 
 seem sustainable
Why are you running the tests at all for building a package? The command make ldmd2 druntime-ldc phobos2-ldc will just build the compiler and standard library, along with the dmd-compatible driver. I assume you are running make alone, which might run the tests too. Add druntime-ldc-debug and phobos2-ldc-debug to the list to get everything that normally comes with an ldc release.
Jan 04 2017
parent reply fredmanglis <fredmanglis gmail.com> writes:
On Wednesday, 4 January 2017 at 19:15:09 UTC, Joakim wrote:
 Why are you running the tests at all for building a package?
Well, as a principle, the Guix package manager runs tests while building the package as part of ensuring that the package was built correctly. To ensure reproducibility of the builds, however, the environment set up cuts away anything that introduces variations, network access being one of the major ones. As such, I need that the tests are run, otherwise the package definition will not be accepted (I can't simply cop-out and disable all the tests). To be clear, the building phase completes successfully, but the testing phase fails, and Guix requires that all the stages pass successfully.
Jan 04 2017
parent reply Joakim <dlang joakim.fea.st> writes:
On Thursday, 5 January 2017 at 06:18:25 UTC, fredmanglis wrote:
 On Wednesday, 4 January 2017 at 19:15:09 UTC, Joakim wrote:
 Why are you running the tests at all for building a package?
Well, as a principle, the Guix package manager runs tests while building the package as part of ensuring that the package was built correctly. To ensure reproducibility of the builds, however, the environment set up cuts away anything that introduces variations, network access being one of the major ones. As such, I need that the tests are run, otherwise the package definition will not be accepted (I can't simply cop-out and disable all the tests). To be clear, the building phase completes successfully, but the testing phase fails, and Guix requires that all the stages pass successfully.
I see, do you run all the ldc tests, including the dmd testsuite? Can you post the patch you're using now, so we can get an idea of what you had to disable? I haven't had many problems running the tests even without network access, I believe they should just keep going. This could be pulled upstream, especially since they seem to be focusing on getting into distros recently: http://forum.dlang.org/thread/o3cdl9$ecs$1 digitalmars.com Maybe we can get a PR in upstream to selectively disable tests that error out in your test environment, which other packagers might want too and might save you the trouble of maintaining a patch yourself. Let me know and I can help you with that.
Jan 04 2017
parent reply fredmanglis <fredmanglis gmail.com> writes:
On Thursday, 5 January 2017 at 07:06:11 UTC, Joakim wrote:
 I see, do you run all the ldc tests, including the dmd 
 testsuite? Can you post the patch you're using now, so we can 
 get an idea of what you had to disable?  I haven't had many 
 problems running the tests even without network access, I 
 believe they should just keep going.
Yes. The system runs all the tests, including dmd-testsuite. As for the patches, there are 2, one for phobos, and one for dmd. I do not see an attach file option, and I don't know if it is acceptable to just dump the patches here: what's a convenient way to send the patches?
Jan 05 2017
parent reply David Nadlinger via digitalmars-d-ldc <digitalmars-d-ldc puremagic.com> writes:
On Thu, Jan 5, 2017 at 5:17 PM, fredmanglis via digitalmars-d-ldc
<digitalmars-d-ldc puremagic.com> wrote:
 As for the patches, there are 2, one for phobos, and one for dmd. I do not
 see an attach file option, and I don't know if it is acceptable to just dump
 the patches here: what's a convenient way to send the patches?
The way to submit them for inclusion would be as a PR at https://github.com/dlang/dmd and https://github.com/dlang/phobos. For just sharing them here, you can either attach them when using the email interface (they'll also show up in the web ui), or just paste them somewhere (Gist, ...) and post the link here. IIRC there is already something like PHOBOS_TEST_ALLOW_NET for Phobos, though? (Not on my dev machine right now.) — David
Jan 05 2017
parent reply fredmanglis <fredmanglis gmail.com> writes:
On Thursday, 5 January 2017 at 16:43:47 UTC, David Nadlinger 
wrote:
 On Thu, Jan 5, 2017 at 5:17 PM, fredmanglis via 
 digitalmars-d-ldc <digitalmars-d-ldc puremagic.com> wrote:
 As for the patches, there are 2, one for phobos, and one for 
 dmd. I do not see an attach file option, and I don't know if 
 it is acceptable to just dump the patches here: what's a 
 convenient way to send the patches?
The way to submit them for inclusion would be as a PR at https://github.com/dlang/dmd and https://github.com/dlang/phobos.
For now, I am not submitting the patches for inclusion, since as the thread notes, they are a shim, to get Guix to pass the tests
 For just sharing them here, you can either attach them when 
 using the
 email interface (they'll also show up in the web ui), or just 
 paste
 them somewhere (Gist, ...) and post the link here.
The patch I made for phobos: https://gist.github.com/fredmanglis/02d5255feba18167aaa6b23942553928#file-ldc1-1-0-disable-phobos-tests-patch The patch I made for dmd-testsuite: https://gist.github.com/fredmanglis/a423c18abad0bfd3b27be49aa722dcc5#file-ldc1-1-0-disable-dmd-tests-patch
 IIRC there is already something like PHOBOS_TEST_ALLOW_NET for 
 Phobos, though? (Not on my dev machine right now.)
Okay. That seems promising. Let me look for that
Jan 05 2017
parent reply Joakim <dlang joakim.fea.st> writes:
On Thursday, 5 January 2017 at 17:31:38 UTC, fredmanglis wrote:
 For just sharing them here, you can either attach them when 
 using the
 email interface (they'll also show up in the web ui), or just 
 paste
 them somewhere (Gist, ...) and post the link here.
The patch I made for phobos: https://gist.github.com/fredmanglis/02d5255feba18167aaa6b23942553928#file-ldc1-1-0-disable-phobos-tests-patch The patch I made for dmd-testsuite: https://gist.github.com/fredmanglis/a423c18abad0bfd3b27be49aa722dcc5#file-ldc1-1-0-disable-dmd-tests-patch
Seem fairly small, but I could see why you wouldn't want to maintain them yourself. If you're up for it, we could probably add a version flag to the test suite to disable these sections for packagers.
 IIRC there is already something like PHOBOS_TEST_ALLOW_NET for 
 Phobos, though? (Not on my dev machine right now.)
Okay. That seems promising. Let me look for that
Never heard of it, may have been removed.
Jan 05 2017
parent reply fredmanglis <fredmanglis gmail.com> writes:
On Thursday, 5 January 2017 at 23:18:36 UTC, Joakim wrote:
 Seem fairly small, but I could see why you wouldn't want to 
 maintain them yourself.  If you're up for it, we could probably 
 add a version flag to the test suite to disable these sections 
 for packagers.
I am up for adding the version flag, though I might have to learn D first.
 IIRC there is already something like PHOBOS_TEST_ALLOW_NET 
 for Phobos, though? (Not on my dev machine right now.)
Okay. That seems promising. Let me look for that
Never heard of it, may have been removed.
Yeah, My searches for this do not seem to be conclusive yet. I will look at it some more though to ensure I am not missing anything.
Jan 06 2017
parent reply Pjotr Prins <pjotr.public12 thebird.nl> writes:
On Friday, 6 January 2017 at 13:39:31 UTC, fredmanglis wrote:
 On Thursday, 5 January 2017 at 23:18:36 UTC, Joakim wrote:
 Seem fairly small, but I could see why you wouldn't want to 
 maintain them yourself.  If you're up for it, we could 
 probably add a version flag to the test suite to disable these 
 sections for packagers.
Similar to StdUnittest in https://github.com/dlang/phobos/pull/5927, I propose we add version(StdNetworkunittest) and/or version(StdEnvUnittest) for those tests that require internet access or access to a directory outside the source tree (e.g. $TMPDIR) as these may not be available in containers and other restrictive build systems. Does that make sense?
Feb 11
parent reply Seb <seb wilzba.ch> writes:
On Sunday, 11 February 2018 at 15:39:18 UTC, Pjotr Prins wrote:
 On Friday, 6 January 2017 at 13:39:31 UTC, fredmanglis wrote:
 On Thursday, 5 January 2017 at 23:18:36 UTC, Joakim wrote:
 Seem fairly small, but I could see why you wouldn't want to 
 maintain them yourself.  If you're up for it, we could 
 probably add a version flag to the test suite to disable 
 these sections for packagers.
Similar to StdUnittest in https://github.com/dlang/phobos/pull/5927, I propose we add version(StdNetworkunittest) and/or version(StdEnvUnittest) for those tests that require internet access or access to a directory outside the source tree (e.g. $TMPDIR) as these may not be available in containers and other restrictive build systems. Does that make sense?
How about the reverse, s.t. not all build scripts need to be updated?
Feb 11
next sibling parent Pjotr Prins <pjotr.public12 thebird.nl> writes:
On Sunday, 11 February 2018 at 19:40:56 UTC, Seb wrote:
 How about the reverse, s.t. not all build scripts need to be 
 updated?
The following would only be for tests that access the internet: version(StdEnvUnittest) { do something with internet enabled } Not that many tests.
Feb 11
prev sibling parent Pjotr Prins <pjotr.public12 thebird.nl> writes:
On Sunday, 11 February 2018 at 19:40:56 UTC, Seb wrote:
 How about the reverse, s.t. not all build scripts need to be 
 updated?
Can we do something like !version(ConstrainedEnvUnittest) { networking.... } i.e., a negative? Or do we need to write version(ConstrainedEnvUnittest) { // empty } else { networking... } everywhere. Maybe version is not the best way to do this.
Feb 11
prev sibling parent reply Thomas Mader <thomas.mader gmail.com> writes:
On Wednesday, 4 January 2017 at 18:40:57 UTC, fredmanglis wrote:
 Hi.

 I'm packaging ldc-1.1.0-beta6 for guix, which does not allow 
 for network access when building the package, leading to 
 errors, and timeouts when the tests are run.

 Is there a way to deactivate such tests, say a flag passed to 
 the build system or such? I would be grateful if you'd point me 
 to one.

 In the meantime, I have had to make use of huge patches to 
 deactivate the tests to make the builds pass, which does not 
 seem sustainable
I am packaging ldc for NixOS and I was having the same problem there. I solved it by putting the tests with special needs into a fixed output derivation. That's how it is called in Nix world. Fixed output derivation's always produce the same output and are therefore allowed to access the network. They are also used to implement all kinds of fetchers to get the source archives from GitHub and the like. guix sure has something equal because it basically is the same system. They even use parts of Nix. Here you can take a look into the current package: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/compilers/ldc/default.nix
Feb 12
parent reply Pjotr Prins <pjotr.public12 thebird.nl> writes:
On Monday, 12 February 2018 at 21:19:35 UTC, Thomas Mader wrote:
 I am packaging ldc for NixOS and I was having the same problem 
 there. I solved it by putting the tests with special needs into 
 a fixed output derivation.
 That's how it is called in Nix world. Fixed output derivation's 
 always produce the same output and are therefore allowed to 
 access the network. They are also used to implement all kinds 
 of fetchers to get the source archives from GitHub and the like.
That is an interesting approach and I will discuss it with others. So an output derivation is an output that gets generated and allows for running it after the installation phase - giving it an opportunity to run tests. It is an interesting approach and works because the store is 'immutable'. Two downsides: (1) is you can not assume there is a network on the build machine. We need to be able to build and test packages in isolation on restricted computers to make sure no one can tamper. (2) it allows pulling software, as you state, which changes the state of test system and makes it potentially non-reproducible. I don't think the Guix folks thing that this is the way forward. Impurity is a big no.
 guix sure has something equal because it basically is the same 
 system. They even use parts of Nix.
Currently Nix and Guix share the build daemon. Otherwise Nix and Guix are different animals altogether. I moved from Nix to Guix a few years back and know both systems intimately.
 Here you can take a look into the current package: 
 https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/compilers/ldc/default.nix
here is mine (almost done, I need to remove some lines): https://gitlab.com/genenetwork/guix/blob/dlang/gnu/packages/ldc.scm#L183 to me the difference is that (if you think away the LISP braces) Guix looks a lot more readable and the logic is clear from the expression (some of it is probably a matter of taste). I also note that the Nix package does not to build shared phobos libs and is not running all ldc tests. But, admittedly, both package descriptions are short and clear.
Feb 15
next sibling parent reply Kagamin <spam here.lot> writes:
On Friday, 16 February 2018 at 06:13:05 UTC, Pjotr Prins wrote:
 Two downsides: (1) is you can not assume there is a network on 
 the build machine. We need to be able to build and test 
 packages in isolation on restricted computers to make sure no 
 one can tamper. (2) it allows pulling software, as you state, 
 which changes the state of test system and makes it potentially 
 non-reproducible. I don't think the Guix folks thing that this 
 is the way forward. Impurity is a big no.
Do you mean phobos curl tests? They use a self-contained test server, should be pure.
Feb 16
parent reply Pjotr Prins <pjotr.public12 thebird.nl> writes:
On Friday, 16 February 2018 at 12:00:18 UTC, Kagamin wrote:
 On Friday, 16 February 2018 at 06:13:05 UTC, Pjotr Prins wrote:
 Two downsides: (1) is you can not assume there is a network on 
 the build machine. We need to be able to build and test 
 packages in isolation on restricted computers to make sure no 
 one can tamper. (2) it allows pulling software, as you state, 
 which changes the state of test system and makes it 
 potentially non-reproducible. I don't think the Guix folks 
 thing that this is the way forward. Impurity is a big no.
Do you mean phobos curl tests? They use a self-contained test server, should be pure.
Those are fine. Current patches to make the build work are https://gitlab.com/genenetwork/guix/blob/dlang/gnu/packages/patches/ldc-1.7.0-disable-phobos-tests.patch Not too many! Remaining problems are with assumptions about socket and root path. I'll submit some patches if I can make it work. The root path can probably be avoided by faking it into the build dir.
Feb 16
parent Thomas Mader <thomas.mader gmail.com> writes:
On Friday, 16 February 2018 at 13:14:22 UTC, Pjotr Prins wrote:
 Those are fine. Current patches to make the build work are

   
 https://gitlab.com/genenetwork/guix/blob/dlang/gnu/packages/patches/ldc-1.7.0-disable-phobos-tests.patch

 Not too many!

 Remaining problems are with assumptions about socket and root 
 path. I'll submit some patches if I can make it work. The root 
 path can probably be avoided by faking it into the build dir.
Solutions to phobos problems already went upstream. Some of them should be getting out with the next major version release for dmd. You could go through the package script for dmd, some of them have comments. The ones I remember which are fixed: root/homedirectory, datetime problems with the path to the timezone db, curl path problems. The remaining problems I am having seem to be Nix related. So you shouldn't have other problems with guix than me with Nix. It might be a good idea if we follow each other on Github or something to prevent double work.
Feb 17
prev sibling next sibling parent Thomas Mader <thomas.mader gmail.com> writes:
On Friday, 16 February 2018 at 06:13:05 UTC, Pjotr Prins wrote:
 That is an interesting approach and I will discuss it with 
 others. So an output derivation is an output that gets 
 generated and allows for running it after the installation 
 phase - giving it an opportunity to run tests. It is an 
 interesting approach and works because the store is 'immutable'.

 Two downsides: (1) is you can not assume there is a network on 
 the build machine. We need to be able to build and test 
 packages in isolation on restricted computers to make sure no 
 one can tamper.
As Kagamin already noted, Phobos isn't running any network tests but dub is in it's tests. But I don't see that as a problem. You always depend on the network for all package builds because you need to get the source from somewhere. If you want to be able to build without Internet access you need to have it cached somewhere. Either on your machine or on a proxy. That's how it works. Is that different with guix?
 (2) it allows pulling software, as you state, which changes the 
 state of test system and makes it potentially non-reproducible. 
 I don't think the Guix folks thing that this is the way 
 forward. Impurity is a big no.
As you already stated a fixed output derivation isn't changing the derivation which uses it because the output is the same all the time. What happens inside the derivation is of no concern to the using derivation as long as it runs through.
 guix sure has something equal because it basically is the same 
 system. They even use parts of Nix.
Currently Nix and Guix share the build daemon. Otherwise Nix and Guix are different animals altogether. I moved from Nix to Guix a few years back and know both systems intimately.
I decided to go with Nix because the community seems bigger and it was the project with the initial idea for such a system. I don't see me moving but would still be interested in your reasons.
 here is mine (almost done, I need to remove some lines):

     
 https://gitlab.com/genenetwork/guix/blob/dlang/gnu/packages/ldc.scm#L183

 to me the difference is that (if you think away the LISP 
 braces) Guix looks a lot more readable and the logic is clear 
 from the expression (some of it is probably a matter of taste). 
 I also note that the Nix package does not to build shared 
 phobos libs and is not running all ldc tests.

 But, admittedly, both package descriptions are short and clear.
Sorry I can't see that. :-) I knew that guix comes with it's own language and if I remember correct I read somewhere that the Nix project lead is not against something like that for Nix too but I am not sure if it is currently an active topic. But I don't think it is. I don't care, for me it would be more important to have such a system for Windows too to be able to have one package manager for the major three OSes. Does guix come with a nice command line interface? Nix used to have multiple commands but with Nix 2.0 all is unified via 'nix <subcommand>'.
Feb 17
prev sibling parent Thomas Mader <thomas.mader gmail.com> writes:
On Friday, 16 February 2018 at 06:13:05 UTC, Pjotr Prins wrote:
 I also note that the Nix package does not to build shared 
 phobos libs and is not running all ldc tests.
Which ones are missing? Why are the shared libs not built?
Feb 17