www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Fetch and pre-build dub dependencies

reply Tamas <user dlang.io> writes:
I have a Dockerfile to build a vibe.d project. Docker has a nice 
caching system so it continues building the docker image from the 
step where it's needed when something changes. This is typically 
at the step of adding my source files, then building the binary.

The problem is that fetching and building vibe dependencies takes 
a lot of time every time some source files of mine changes.

I can pre-fetch dub dependencies, although it's a bit cumbersome 
as fetch doesn't load dependencies of the given package.

     RUN dub fetch vibe-d --cache=system
     RUN dub fetch vibe-core --cache=system
     RUN dub fetch eventcore --cache=system
     ....

But there's no easy way to build these dependencies before I add 
my project source files to the docker image. If I could do that 
the docker build process would be tremendously faster, as it 
would be simply skipped automatically by the Docker cache system. 
Currently 99% of the time spent on building the dependencies when 
i am creating a docker image. (I have to use ldc for ARM, which 
is slower.)

Is there an easy way to fetch and prebuild a dub package 
dependencies?

something like "dub fetch-and-build vibe-d --recursive 
--cache=system"
Mar 06 2018
parent reply Seb <seb wilzba.ch> writes:
On Tuesday, 6 March 2018 at 18:03:34 UTC, Tamas wrote:
 I have a Dockerfile to build a vibe.d project. Docker has a 
 nice caching system so it continues building the docker image 
 from the step where it's needed when something changes. This is 
 typically at the step of adding my source files, then building 
 the binary.

 The problem is that fetching and building vibe dependencies 
 takes a lot of time every time some source files of mine 
 changes.

 I can pre-fetch dub dependencies, although it's a bit 
 cumbersome as fetch doesn't load dependencies of the given 
 package.

     RUN dub fetch vibe-d --cache=system
     RUN dub fetch vibe-core --cache=system
     RUN dub fetch eventcore --cache=system
     ....

 But there's no easy way to build these dependencies before I 
 add my project source files to the docker image. If I could do 
 that the docker build process would be tremendously faster, as 
 it would be simply skipped automatically by the Docker cache 
 system. Currently 99% of the time spent on building the 
 dependencies when i am creating a docker image. (I have to use 
 ldc for ARM, which is slower.)

 Is there an easy way to fetch and prebuild a dub package 
 dependencies?

 something like "dub fetch-and-build vibe-d --recursive 
 --cache=system"
For run.dlang.io, I fetch all dependencies and build them into the Docker image. That has the advantage that executions with dub packages are a lot faster. It's just dub fetch and dub build though. Take a look: https://github.com/dlang-tour/core-exec/blob/master/Dockerfile dub fetch also fetches the dependencies of the package. The docker images don't have internet access on run.dlang.io, so it works fine for me. Maybe all you need is to disable the registry lookup?
Mar 06 2018
parent Tamas <user dlang.io> writes:
On Wednesday, 7 March 2018 at 04:28:11 UTC, Seb wrote:
 For run.dlang.io, I fetch all dependencies and build them into 
 the Docker image. That has the advantage that executions with 
 dub packages are a lot faster.
 It's just dub fetch and dub build though. Take a look:

 https://github.com/dlang-tour/core-exec/blob/master/Dockerfile
Thank you for showing your solution! Using dub --single is neat and works well for this!
Mar 06 2018