digitalmars.D - dub with nix (issue introduced by dub >= dub-v1.31.0)
- Ralph Amissah (136/136) May 26 2023 My offline D project with locally downloaded subpackage directory
- Richard (Rikki) Andrew Cattermole (6/6) May 26 2023 The most likely change to result in this is:
- Ralph Amissah (23/27) May 26 2023 Rikki, thanks for your response. Something is changed from v1.31.0 I
- WebFreak001 (6/15) May 26 2023 thanks for the response, this seems to be a bug. I'll investigate
- Ralph Amissah (6/11) May 30 2023 Thank you. I am grateful for your work and that of others in the D
- Ralph (13/32) Jul 06 2023 Grateful for any updates on this problem.
- WebFreak001 (3/17) Jul 06 2023 ah sorry no progress on it yet, but the upstream issue is here:
- Ralph (24/26) Jul 08 2023 mea culpa (I have had another look)
- Vladimir Panteleev (4/5) Jul 08 2023 Also have you tried `--temp-build`? With it Dub should not try to
- Ralph (11/16) Jul 08 2023 just did out of curiosity, different error:
My offline D project with locally downloaded subpackage directory trees has built flawlessly with nix tools for previous versions of dub from dub-v1.23.0 through dub-v1.30.0 which is the version currently in nixpkgs. However, a change to dub introduced in 1.31.0 results in my project failing to build using dub with nix tools. The build attempt results in the nix not finding build directory paths as before (the build directory within nixpkgs has been affected as now it attempts to build in a write protected area (it ends up in what nix calls a "homeless shelter"!) I am hoping that it might be easy to guess what the change that would have such an effect is likely to be... and that it can be fixed or reverted. https://github.com/dlang/dub/releases/tag/v1.31.0 To give an idea of my current dub build dependencies etc. I include a copy of my dub.json (below) Incidentally, within nix I have flakes with nixpkgs overlays which make it easy to play with various version changes, and modify the build process for nixpkgs for various things D (ldc2 dub dtools and the like). Dub builds fine and passes most if not all of its tests in nix (in the current nixpkgs build several of the dub tests remain removed for having failed in the past). I would be happy to share these, as they may reduce the hurdle for anyone using D who is interested in experimenting with nix. { "authors": [ "Ralph Amissah" ], "copyright": "2015 - 2023 Ralph Amissah", "name": "spine", "version": "0.12.0", "description": "an object-centric sisu-like document parser", "homepage": "https://sisudoc.org", "license": "AGPL-3.0+", "targetPath": "./bin", "sourcePaths": [ "./src/doc_reform" ], "stringImportPaths": [ "./views" ], "buildRequirements": [ "allowWarnings" ], "targetType": "executable", "buildTypes": { "dmd": { "dflags": [ "-J=./views", "-I=./src/doc_reform" ], "buildOptions": [ "inline", "verbose" ], "buildRequirements": [ "allowWarnings" ] }, "ldc2": { "dflags": [ "-O2", "-J=./views", "-I=./src/doc_reform" ], "buildOptions": [ "optimize", "verbose" ], "buildRequirements": [ "allowWarnings" ] }, "ldmd2": { "dflags": [ "-O2", "-boundscheck=on", "-J=./views", "-I=./src/doc_reform" ], "buildOptions": [ "optimize", "verbose" ], "buildRequirements": [ "allowWarnings" ] } }, "dependencies": { "spine:d2sqlite3": "*", "spine:imageformats": "*", "spine:dyaml": "*" }, "subPackages": [ { "name": "d2sqlite3", "description": "A thin wrapper around SQLite 3", "homepage": "https://github.com/dlang-community/d2sqlite3", "authors": [ "Nicolas Sicard", "Other contributors: see Github repo" ], "copyright": "Copyright 2011-18 Nicolas Sicard", "license": "BSL-1.0", "sourcePaths": [ "./src/ext_depends/d2sqlite3/source" ], "importPaths": [ "./src/ext_depends/d2sqlite3/source" ], "configurations": [ { "name": "d2sqlite3", "targetType": "library", "systemDependencies": "SQLite version >= 3.8.7", "libs": [ "sqlite3" ], "excludedSourceFiles": [ "source/tests.d" ] } ] }, { "name": "imageformats", "description": "Decoders for PNG, TGA, BMP, JPEG and encoders for PNG, TGA, BMP.", "authors": [ "Tero Hänninen" ], "license": "BSL-1.0", "sourcePaths": [ "./src/ext_depends/imageformats" ], "importPaths": [ "./src/ext_depends/imageformats" ] }, { "name": "dyaml", "description": "YAML parser and emitter", "authors": [ "Ferdinand Majerech", "Cameron \"Herringway\" Ross" ], "license": "BSL-1.0", "homepage": "https://github.com/dlang-community/D-YAML", "copyright": "Copyright © 2011-2018, Ferdinand Majerech", "sourcePaths": [ "./src/ext_depends/D-YAML/source", "./src/ext_depends/tinyendian/source" ], "importPaths": [ "./src/ext_depends/D-YAML/source", "./src/ext_depends/tinyendian/source" ], "preGenerateCommands": [ "rm -rf ./src/ext_depends/D-YAML/examples", "rm -rf ./src/ext_depends/D-YAML/testsuite" ] } ], "configurations": [ { "name": "default", "targetName": "spine" }, { "name": "ldc2", "targetName": "spine-ldc" }, { "name": "ldmd2", "targetName": "spine-ldc" }, { "name": "dmd", "targetName": "spine-dmd" } ] }
May 26 2023
The most likely change to result in this is: https://github.com/dlang/dub/pull/2542 This changed the location for the cache of builds, which is likely the problem you are facing. You could try ``--cache=local`` when building, that should limit it to the directory of the root package you are building.
May 26 2023
Rikki, thanks for your response. Something is changed from v1.31.0 I hope for a fix or workaround. Unfortunately ``--cache=local`` did not solve the problem. using ``--cache=local`` : dub >= v1.31.0 build breaks (trying to build a subpackage, happens to be dyaml i believe as the first sub-package it tries to build) > Error /homeless-shelter: Permission denied dub <= v1.30.0 build is fine So, more generally: On Fri, May 26, 2023 at 11:28 AM Richard (Rikki) Andrew Cattermole via Digitalmars-d <digitalmars-d puremagic.com> wrote:The most likely change to result in this is: https://github.com/dlang/dub/pull/2542 This changed the location for the cache of builds, which is likely the problem you are facing.- Something changed from v1.31.0 to affect building with nix (and that may well be it). - I hope for a fix or workaround (that allows me to continue to use dub with nix). - On IVY and the future direction of D, I like the sound of this, who wouldn't it sounds like a great promise: "Our enthusiasm is high, and we're ready to get going. I think you'll like where we're headed." - On the other hand, I have been extremely content with what I have been able to do with the D and nix ecosystems as they have been, so, I am 100% with you provided I can continue to do what I have been doing. - Otherwise, belatedly, please note my first MAJOR "Gripe and Wish"; if we find an elegant way around it, and no problem.
May 26 2023
On Friday, 26 May 2023 at 18:53:36 UTC, Ralph Amissah wrote:
Rikki, thanks for your response. Something is changed from
v1.31.0 I hope for a fix or workaround.
Unfortunately ``--cache=local`` did not solve the problem.
using ``--cache=local`` :
dub >= v1.31.0 build breaks (trying to build a subpackage,
happens to
be dyaml i believe as the first sub-package it tries to build)
> Error /homeless-shelter: Permission denied
[...]
thanks for the response, this seems to be a bug. I'll investigate
this in dub and ensure --cache=local works, as well as giving the
option to always use cache local by default, so that users with
read-only filesystems can continue to use dub, even though
filesystem usage will be much higher.
May 26 2023
Ralph Amissah <ralph.amissah gmail.com> On Fri, May 26, 2023 at 4:51 PM WebFreak001 via Digitalmars-d <digitalmars-d puremagic.com> wrote:thanks for the response, this seems to be a bug. I'll investigate this in dub and ensure --cache=local works, as well as giving the option to always use cache local by default, so that users with read-only filesystems can continue to use dub, even though filesystem usage will be much higher.Thank you. I am grateful for your work and that of others in the D (and nix and wider FOSS) community(ies) I look forward to being able to report that (build options are preserved and) everything is once again as it should be.
May 30 2023
On Friday, 26 May 2023 at 20:48:41 UTC, WebFreak001 wrote:On Friday, 26 May 2023 at 18:53:36 UTC, Ralph Amissah wrote:Grateful for any updates on this problem. I have been following and testing subsequent dub releases with Nix flakes (nix build) (on NixOS). Dub 1.33.0 was released too soon after this report to take it into account; Dub 1.33.1 released July 2nd still does not address the issue. I would just like to be re-assured that the issue is not forgotten. https://github.com/NixOS/nixpkgs/pull/226813 It would be wonderful to know that it has been sorted. thanks, RalphRikki, thanks for your response. Something is changed from v1.31.0 I hope for a fix or workaround. Unfortunately ``--cache=local`` did not solve the problem. using ``--cache=local`` : dub >= v1.31.0 build breaks (trying to build a subpackage, happens to be dyaml i believe as the first sub-package it tries to build) > Error /homeless-shelter: Permission denied [...]thanks for the response, this seems to be a bug. I'll investigate this in dub and ensure --cache=local works, as well as giving the option to always use cache local by default, so that users with read-only filesystems can continue to use dub, even though filesystem usage will be much higher.
Jul 06 2023
On Thursday, 6 July 2023 at 13:28:38 UTC, Ralph wrote:On Friday, 26 May 2023 at 20:48:41 UTC, WebFreak001 wrote:ah sorry no progress on it yet, but the upstream issue is here: https://github.com/dlang/dub/issues/2654[...]Grateful for any updates on this problem. I have been following and testing subsequent dub releases with Nix flakes (nix build) (on NixOS). Dub 1.33.0 was released too soon after this report to take it into account; Dub 1.33.1 released July 2nd still does not address the issue. I would just like to be re-assured that the issue is not forgotten. https://github.com/NixOS/nixpkgs/pull/226813 It would be wonderful to know that it has been sorted. thanks, Ralph
Jul 06 2023
mea culpa (I have had another look) Yes there was a change in the behavior of dub between v1.30.0 and v1.31.0 onward which (on NixOS) resulted in a build problem with attempting to write to a non-writable /homeless-shelter directory (on NixOS, possibly using nix generally I have not checked). However, the change does not really matter as there is a simple solution within nix which is in the buildPhase to make sure that the directory HOME exists and/or is set to a writable directory. export HOME=$(pwd) which I just found here: - https://github.com/NixOS/nix/issues/670 (It is not for want of having searched for the like (and not found such) earlier though). I would not bother further digging into this matter of what changed within dub, at least on NixOS it does not seem to matter provided you know what to do if caught by it. On the positive side I can confirm that all versions of dub including latest dub 1.33.1, build in nix and pass almost all tests (several were removed to make dub work with earlier nixpkg builds). will update: On Thursday, 6 July 2023 at 15:17:49 UTC, WebFreak001 wrote:Thank you for your feedback, Ralphhttps://github.com/NixOS/nixpkgs/pull/226813
Jul 08 2023
On Saturday, 8 July 2023 at 18:24:23 UTC, Ralph wrote:export HOME=$(pwd)Also have you tried `--temp-build`? With it Dub should not try to write to `$HOME` either, I think, just the current project's directory.
Jul 08 2023
On Saturday, 8 July 2023 at 19:22:28 UTC, Vladimir Panteleev wrote:On Saturday, 8 July 2023 at 18:24:23 UTC, Ralph wrote:just did out of curiosity, different error: it appears to build but the result of my build is not available to me ... i guess unless I take some additional step that I have never had to... but thanks. making sure HOME directory is set (in the nix "buildPhase" or "preBuild" as I ended up doing) to wherever nix chooses to do its build by default (pwd) is a better solution (my guess: it seems that somehow in dub HOME has been lost and some action of dub is now taken on root following an empty HOME)export HOME=$(pwd)Also have you tried `--temp-build`? With it Dub should not try to write to `$HOME` either, I think, just the current project's directory.
Jul 08 2023