www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Git, the D package manager

reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 2 February 2015 at 05:23:52 UTC, Daniel Murphy wrote:

 "Vladimir Panteleev"  wrote in message 
 news:viqwfixznbdbdwvhavuk forum.dlang.org...


 I don't use Dub


 You really should!  I put it off for months and months but I'm 
 quite happy with it now.


Even if I had faith that dub was a perfectly polished piece of 
software, it doesn't solve any problems I have with building D 
programs, and in fact would make said task more complicated. 
Here's why.

1. rdmd

rdmd is a simple and effective way to build D programs, and I'm 
sad to see its use declining.

rdmd leverages two things: D's module system, and the compiler's 
import search path. Dub's design seems to ignore both of the 
above.

1a. rdmd and D's module system:

When you run `dmd -o- program.d`, the compiler will automatically 
read all modules imported by your program, and their imports, and 
so on. It does so by searching the filesystem across its search 
path for matches which correspond with D's module system, and 
only reads those files that are needed.

rdmd leverages this by collecting (and caching) the list of 
modules used in the program, and passing that list to the 
compiler. The compiler will then compile no more and no less than 
the exact set of modules transitively imported by the program.

In contrast, Dub's default modus operandi is to blindly send to 
the compiler all *.d files found in the "src" folder, whether 
they're actually used or not. Not only can this be slower if not 
all modules are always used, but it also won't work if the source 
code contains multiple entry points, forcing you to write 
complicated configuration files (i.e. do the computer's work for 
it).

1b. rdmd and D's search path

rdmd does not have any additional parameters to set up for where 
it needs to look for source files, because it relies on the 
compiler's search mechanism. Thus, if you can build your program 
with rdmd, "dmd -o- program" will succeed, and usually vice versa.

In contrast, Dub builds its own search path using its JSON 
configuration files, and has no equivalent of "dmd -o-".

There is no simple way to syntax-check just one file in a project 
when using Dub. I rely on this a lot in my workflow - I 
configured a syntax-check key in my editor, which I use almost as 
often as I save. A syntax check (dmd -o-) is much faster than a 
full build, as it skips parsing other parts of the project, code 
generation, and linking.

2. git

I've found that git's submodule feature is an excellent way to 
manage dependencies across D libraries, and fits really well with 
D's package system. For clarity to readers not too familiar with 
Git, I'll explain by example:

2a. Directory structure

If you have a library "fruit" with the modules "fruit.apple" and 
"fruit.banana", create a git repository (in a directory called 
"fruit") with the files apple.d and banana.d.

Now, in your project (e.g. "orchard"), add a symlink called 
"fruit" pointing to the repository:

~/
|- fruit/
|  |- .git/
|  |- apple.d
|  '- banana.d
'- orchard/
    |- .git/
    |- fruit -> ~/fruit/
    '- tree.d

tree.d can now import the "fruit.apple" module without any 
additional compiler switches. `rdmd tree` will just work. This is 
because the modules of the "fruit" library are placed at the 
repository root, and the repository's directory name matches with 
the library's package name, so when the compiler will look for 
the "fruit.apple" module, it will find it at "fruit/apple.d".

2b. Distribution

Upload the "fruit" repository to GitHub. In your "orchard" 
project, delete the symlink, and run:

git submodule add https://github.com/You/fruit fruit

This will clone the repository from GitHub, create a .gitmodules 
file, and add an entry to the git index. If you commit all these, 
and put "orchard" on GitHub too, someone can run the command:

git clone --recursive https://github.com/You/orchard

... to get the orchard project, and its dependencies (the fruit 
library).

2c. Versioning

Git stores the exact commit of each submodule in the parent 
repository.

So, if you make a commit in orchard/fruit/, and run `git status` 
in orchard/, git will show you that the "fruit" submodule has 
been modified.

Most importantly, this means that any breaking change in the 
"fruit" library will never affect existing projects which use it, 
since they will continue to use the registered commit which was 
known to work.

This gives you dependency versioning with commit granularity - 
you can hardly ask for something better - all without messing 
with configuration files.

---

The one thing I wish git had is symlink support for Windows. 
Windows has symlinks and directory junctions, but msysgit still 
doesn't support them.

For a practical instance of the above example, see Digger:

https://github.com/CyberShadow/Digger

Note the simple build instructions:

https://github.com/CyberShadow/Digger#building

2d. Git vs. Dub

Unfortunately, the above-described approach is not compatible 
with Dub:

- Dub packages are generally expected to have their source code 
in a "src" subdirectory, although you can set the source 
directory to "." in the configuration file.

- When cloning repositories, dub does not preserve the 
repository's directory name (so e.g. fruit will be cloned to 
~/.dub/fruit-1.0.0/).

Somebody has created a Dub package for my library (excluding 
certain packages, due to point 1a above), and the other day 
someone complained that it doesn't work with Dscanner, because of 
the above issue - the module path "ae.dir.module" does not 
correspond to the filesystem path "ae-1.0.1/dir/module.d".

So, in order to start using Dub, I'd need to:

- restructure the directory layout of my library (breaking change)
- update all projects which use this library to use Dub instead
- give up quick syntax checking
- give up commit-granularity versioning
- begin maintaining JSON configuration files
- begin versioning libraries by hand
- install Dub on all my computers, servers, and virtual machines

No thanks.

I could invest time in improving Dub to fix or ameliorate some of 
the above points, but I don't see a compelling reason to. In 
fact, I think we should integrate rdmd into dmd - dmd clearly 
already knows which source files participate in compilation, as 
all rdmd does is basically take dmd's output and feed it back to 
it. This will greatly speed up compilation, too.

Change my view.
Feb 02 2015
next sibling parent reply "Daniel Kozak" <kozzi11 gmail.com> writes:
On Monday, 2 February 2015 at 08:09:39 UTC, Vladimir Panteleev 
wrote:

 1a. rdmd and D's module system:

 When you run `dmd -o- program.d`, the compiler will 
 automatically read all modules imported by your program, and 
 their imports, and so on. It does so by searching the 
 filesystem across its search path for matches which correspond 
 with D's module system, and only reads those files that are 
 needed.

 rdmd leverages this by collecting (and caching) the list of 
 modules used in the program, and passing that list to the 
 compiler. The compiler will then compile no more and no less 
 than the exact set of modules transitively imported by the 
 program.

 In contrast, Dub's default modus operandi is to blindly send to 
 the compiler all *.d files found in the "src" folder, whether 
 they're actually used or not. Not only can this be slower if 
 not all modules are always used, but it also won't work if the 
 source code contains multiple entry points, forcing you to 
 write complicated configuration files (i.e. do the computer's 
 work for it).


And this is reason why rdmd suck (in my case).
Example:

return 
cast(EntityManagerFactory)Object.factory("cz.dlang.ddpa.EntityM
nagerFactoryImpl"); 
// this does not work with rdmd but works ok with dub
Feb 02 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 2 February 2015 at 08:46:40 UTC, Daniel Kozak wrote:

 And this is reason why rdmd suck (in my case).
 Example:

 return 
 cast(EntityManagerFactory)Object.factory("cz.dlang.ddpa.EntityManagerFactoryImpl");


Object.factory is a misfeature. It causes ALL classes and their 
virtual methods and everything that they refer, and so on, to be 
built into the resulting binary, regardless if they're used or 
not, and regardless if you use Object.factory or not.


 // this does not work with rdmd but works ok with dub


Fix: static import cz.dlang.ddpa;
Feb 02 2015
parent "Daniel Kozak" <kozzi11 gmail.com> writes:
On Monday, 2 February 2015 at 08:52:29 UTC, Vladimir Panteleev 
wrote:

 On Monday, 2 February 2015 at 08:46:40 UTC, Daniel Kozak wrote:

 And this is reason why rdmd suck (in my case).
 Example:

 return 
 cast(EntityManagerFactory)Object.factory("cz.dlang.ddpa.EntityManagerFactoryImpl");


 Object.factory is a misfeature. It causes ALL classes and their 
 virtual methods and everything that they refer, and so on, to 
 be built into the resulting binary, regardless if they're used 
 or not, and regardless if you use Object.factory or not.


 // this does not work with rdmd but works ok with dub


 Fix: static import cz.dlang.ddpa;


Not ideal solution, but probbably doable. ATM I have xml file 
with cz.dlang.ddpa.EntityManagerFactoryImpl so maybe I can make 
it .d file
Feb 02 2015
prev sibling next sibling parent reply Joseph Rushton Wakeling via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 02/02/15 09:09, Vladimir Panteleev via Digitalmars-d wrote:

 2c. Versioning

 Git stores the exact commit of each submodule in the parent repository.

 So, if you make a commit in orchard/fruit/, and run `git status` in orchard/,
 git will show you that the "fruit" submodule has been modified.

 Most importantly, this means that any breaking change in the "fruit" library
 will never affect existing projects which use it, since they will continue to
 use the registered commit which was known to work.

 This gives you dependency versioning with commit granularity - you can hardly
 ask for something better - all without messing with configuration files.


This is both a benefit and a problem.

It's a benefit to the developer, who can specify exactly which version of the 
dependency they want used.  It's not necessarily good for the downstream user, 
because it requires that the developer actively maintain the dependencies.

Scenario: a dependency has a security hole that gets patched.  If the dub 
package is updated, all applications using that dub package will automatically 
have that update available next time they are built.  With git submodules, the 
developer has to recognize that update has happened and tweak the git submodule 
in the master project, and others downstream have to realize that has happened.

Git submodules are great for precise control of dependencies, but there are 
circumstances where it's preferable for upstream to be able to release new 
versions without them being filtered in this way.  I'd readily use git 
submodules in scenario where I controlled all of the code (including what was
in 
the submodules), but not where I had external dependencies.


 2d. Git vs. Dub

 Unfortunately, the above-described approach is not compatible with Dub:

 - Dub packages are generally expected to have their source code in a "src"
 subdirectory, although you can set the source directory to "." in the
 configuration file.


Actually, I think it's 'source'.  But either way, it's just enforcing a good 
layout practice, that can be reworked with the config file.  There is however 
some benefit to imposing a standardized project layout.


 - When cloning repositories, dub does not preserve the repository's directory
 name (so e.g. fruit will be cloned to ~/.dub/fruit-1.0.0/).


Necessary in order to maintain multiple versions of a package in order to 
satisfy different applications' dependencies, surely?  You have a similar 
situation with different versions of shared libraries installed on any UNIX
system.
Feb 02 2015
next sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 2 February 2015 at 08:58:38 UTC, Joseph Rushton 
Wakeling wrote:

 Scenario: a dependency has a security hole that gets patched.  
 If the dub package is updated, all applications using that dub 
 package will automatically have that update available next time 
 they are built.


Is that so? Won't a security fix entail a version bump, requiring 
a change in the requirements file of the parent project? Also, 
does Dub really check for updated versions of libraries online, 
every time a project is built?


 - When cloning repositories, dub does not preserve the 
 repository's directory
 name (so e.g. fruit will be cloned to ~/.dub/fruit-1.0.0/).


 Necessary in order to maintain multiple versions of a package 
 in order to satisfy different applications' dependencies, 
 surely?  You have a similar situation with different versions 
 of shared libraries installed on any UNIX system.


No, it is not necessary. The directory layout could be 
~/.dub/fruit-1.0.0/fruit/...
Feb 02 2015
next sibling parent reply "Mathias LANG" <geod24 gmail.com> writes:
On Monday, 2 February 2015 at 09:03:56 UTC, Vladimir Panteleev 
wrote:

 Is that so? Won't a security fix entail a version bump, 
 requiring a change in the requirements file of the parent 
 project? Also, does Dub really check for updated versions of 
 libraries online, every time a project is built?


It does.
You have a broad range of options for specifying which version to 
use.
http://code.dlang.org/package-format#version-specs
If you use Semver correctly, it's a great benefit.


 - When cloning repositories, dub does not preserve the 
 repository's directory
 name (so e.g. fruit will be cloned to ~/.dub/fruit-1.0.0/).


 Necessary in order to maintain multiple versions of a package 
 in order to satisfy different applications' dependencies, 
 surely?  You have a similar situation with different versions 
 of shared libraries installed on any UNIX system.


 No, it is not necessary. The directory layout could be 
 ~/.dub/fruit-1.0.0/fruit/...


What would be the advantage ? You should not deal with ~/.dub/.
Note, you can specify local packages:
"fruits": { "path": "fruits" }
Works well with git submodules.
Feb 02 2015
next sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 2 February 2015 at 09:25:31 UTC, Mathias LANG wrote:

 On Monday, 2 February 2015 at 09:03:56 UTC, Vladimir Panteleev 
 wrote:

 Is that so? Won't a security fix entail a version bump, 
 requiring a change in the requirements file of the parent 
 project? Also, does Dub really check for updated versions of 
 libraries online, every time a project is built?


 It does.


It actually does a roundtrip every time you build your project???


 You have a broad range of options for specifying which version 
 to use.
 http://code.dlang.org/package-format#version-specs
 If you use Semver correctly, it's a great benefit.


In that case, why is it becoming popular to add 
dub.selections.json as a versioned file to the repository?


 No, it is not necessary. The directory layout could be 
 ~/.dub/fruit-1.0.0/fruit/...


 What would be the advantage ?


The directory structure would correspond with the package 
structure.

As is required by the language specification:

https://github.com/Hackerpilot/DCD/issues/185


 You should not deal with ~/.dub/.


You have to, if you want to use tools that need to read your 
source code other than dub.


 Note, you can specify local packages:
 "fruits": { "path": "fruits" }
 Works well with git submodules.


I don't see what that would do to help. Would this tell dub that 
there is a package called "fruits" in a directory called 
"fruits"? Because the compiler already assumes that already.
Feb 02 2015
parent Mike Parker <aldacron gmail.com> writes:
On 2/2/2015 6:34 PM, Vladimir Panteleev wrote:

 On Monday, 2 February 2015 at 09:25:31 UTC, Mathias LANG wrote:

 On Monday, 2 February 2015 at 09:03:56 UTC, Vladimir Panteleev wrote:

 Is that so? Won't a security fix entail a version bump, requiring a
 change in the requirements file of the parent project? Also, does Dub
 really check for updated versions of libraries online, every time a
 project is built?


 It does.


 It actually does a roundtrip every time you build your project???


No. It requires 'dub upgrade'.
Feb 02 2015
prev sibling parent reply "ponce" <contact gam3sfrommars.fr> writes:
On Monday, 2 February 2015 at 09:25:31 UTC, Mathias LANG wrote:

 On Monday, 2 February 2015 at 09:03:56 UTC, Vladimir Panteleev 
 wrote:

 Is that so? Won't a security fix entail a version bump, 
 requiring a change in the requirements file of the parent 
 project? Also, does Dub really check for updated versions of 
 libraries online, every time a project is built?


 It does.
 You have a broad range of options for specifying which version 
 to use.
 http://code.dlang.org/package-format#version-specs
 If you use Semver correctly, it's a great benefit.


Pretty much. If you don't use version ranges, you fall into the 
diamond dependency problem.

A => B => C v1.2.3
A => D => C v1.3.6

Even if C v1.2.3 and v1.3.6 are API-compatible, you can't build A 
if you don't control both B and C.

So, version ranges are necessary for an ecosystem of libraries.
Feb 02 2015
parent "ponce" <contact gam3sfrommars.fr> writes:
On Monday, 2 February 2015 at 16:56:56 UTC, ponce wrote:

 A => B => C v1.2.3
 A => D => C v1.3.6

 Even if C v1.2.3 and v1.3.6 are API-compatible, you can't build 
 A if you don't control both B and C.

 So, version ranges are necessary for an ecosystem of libraries.


Erratum: if you don't control both B and D*
Feb 02 2015
prev sibling parent reply "Joseph Rushton Wakeling" <joseph.wakeling webdrake.net> writes:
On Monday, 2 February 2015 at 09:03:56 UTC, Vladimir Panteleev 
wrote:

 On Monday, 2 February 2015 at 08:58:38 UTC, Joseph Rushton 
 Wakeling wrote:

 Scenario: a dependency has a security hole that gets patched.  
 If the dub package is updated, all applications using that dub 
 package will automatically have that update available next 
 time they are built.


 Is that so? Won't a security fix entail a version bump, 
 requiring a change in the requirements file of the parent 
 project? Also, does Dub really check for updated versions of 
 libraries online, every time a project is built?


Well, as long as the requirements are expressed in the form,

     "package-name": ">=1.2.3"

then there shouldn't be any problem with a version bump from the 
upstream application.  Whether dub actually does check for 
updates I'm not sure, but it _could_.  Whereas with git 
submodules, you really are stuck with one version and one only.



 - When cloning repositories, dub does not preserve the 
 repository's directory
 name (so e.g. fruit will be cloned to ~/.dub/fruit-1.0.0/).


 Necessary in order to maintain multiple versions of a package 
 in order to satisfy different applications' dependencies, 
 surely?  You have a similar situation with different versions 
 of shared libraries installed on any UNIX system.


 No, it is not necessary. The directory layout could be 
 ~/.dub/fruit-1.0.0/fruit/...


I don't follow your point here.  What's the meaningful difference 
between what dub does and what you suggest?
Feb 02 2015
next sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 2 February 2015 at 10:09:00 UTC, Joseph Rushton 
Wakeling wrote:

 Well, as long as the requirements are expressed in the form,

     "package-name": ">=1.2.3"


This will allow Dub to pick a new major version with incompatible 
changes, no?

Also, see my reply to Mathias regarding dub.selections.json.


 No, it is not necessary. The directory layout could be 
 ~/.dub/fruit-1.0.0/fruit/...


 I don't follow your point here.  What's the meaningful 
 difference between what dub does and what you suggest?


As I said in my reply to Mathias, what dub does breaks the module 
path and file path consistency when modules/subpackages lie in 
the repository root.
Feb 02 2015
next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On 2/2/2015 7:13 PM, Vladimir Panteleev wrote:

 On Monday, 2 February 2015 at 10:09:00 UTC, Joseph Rushton Wakeling wrote:

 Well, as long as the requirements are expressed in the form,

     "package-name": ">=1.2.3"


 This will allow Dub to pick a new major version with incompatible
 changes, no?


"~>1.2.3"
This will constrain it to the range of 1.2.3 ... 1.3.0.
Feb 02 2015
parent "ponce" <contact gam3sfrommars.fr> writes:
On Monday, 2 February 2015 at 13:10:56 UTC, Mike Parker wrote:

 On 2/2/2015 7:13 PM, Vladimir Panteleev wrote:

 On Monday, 2 February 2015 at 10:09:00 UTC, Joseph Rushton 
 Wakeling wrote:

 Well, as long as the requirements are expressed in the form,

    "package-name": ">=1.2.3"


 This will allow Dub to pick a new major version with 
 incompatible
 changes, no?


 "~>1.2.3"
 This will constrain it to the range of 1.2.3 ... 1.3.0.


Better yet, "~>1.2" will constrain from 1.2.0 to 2.0.0 which 
means it will stop updating when incompatible changes are 
released.
Feb 02 2015
prev sibling next sibling parent "ponce" <contact gam3sfrommars.fr> writes:
On Monday, 2 February 2015 at 10:13:27 UTC, Vladimir Panteleev 
wrote:

 On Monday, 2 February 2015 at 10:09:00 UTC, Joseph Rushton 
 Wakeling wrote:

 Well, as long as the requirements are expressed in the form,

    "package-name": ">=1.2.3"


 This will allow Dub to pick a new major version with 
 incompatible changes, no?


Yes, I would advise never to use >=, especially for a library.
http://p0nce.github.io/d-idioms/#Never-use-%3E=-for-dependencies
Feb 02 2015
prev sibling next sibling parent Joseph Rushton Wakeling via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 02/02/15 11:13, Vladimir Panteleev via Digitalmars-d wrote:

 On Monday, 2 February 2015 at 10:09:00 UTC, Joseph Rushton Wakeling wrote:

 Well, as long as the requirements are expressed in the form,

     "package-name": ">=1.2.3"


 This will allow Dub to pick a new major version with incompatible changes, no?


Sorry, I was taking for granted that you'd understand that placing an upper 
bound on versions was also possible.  Thanks to everyone who chipped in with 
examples.


 As I said in my reply to Mathias, what dub does breaks the module path and file
 path consistency when modules/subpackages lie in the repository root.


Well, frankly, I think putting modules or subpackages in the repository root is 
a bad idea.  I'm not sure why I should mind that dub enforces not doing such
things.
Feb 02 2015
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-02-02 11:13, Vladimir Panteleev wrote:


 As I said in my reply to Mathias, what dub does breaks the module path
 and file path consistency when modules/subpackages lie in the repository
 root.


So it's the default behavior that you don't like? I use a similar code 
structure as you (except the submodules) in all my dub projects. I don't 
have a src/source directory.

-- 
/Jacob Carlborg
Feb 04 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Wednesday, 4 February 2015 at 19:04:52 UTC, Jacob Carlborg 
wrote:

 On 2015-02-02 11:13, Vladimir Panteleev wrote:


 As I said in my reply to Mathias, what dub does breaks the 
 module path
 and file path consistency when modules/subpackages lie in the 
 repository
 root.


 So it's the default behavior


Is it a default if you can't change it? Or can it be changed?


 that you don't like?


It is not a question of preference, it is a question of the 
behavior being incompatible with certain tools and workflows.
Feb 04 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-02-05 05:43, Vladimir Panteleev wrote:


 Is it a default if you can't change it? Or can it be changed?


It can be changed. It's the "mainSourceFile" field [1].


 It is not a question of preference, it is a question of the behavior
 being incompatible with certain tools and workflows.


Sure it is, since this can be configured.

[1] http://code.dlang.org/package-format

-- 
/Jacob Carlborg
Feb 07 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Saturday, 7 February 2015 at 14:36:19 UTC, Jacob Carlborg 
wrote:

 On 2015-02-05 05:43, Vladimir Panteleev wrote:


 Is it a default if you can't change it? Or can it be changed?


 It can be changed. It's the "mainSourceFile" field [1].


I don't see how this setting is at all relevant to the problem at 
hand.
Feb 07 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-02-07 20:17, Vladimir Panteleev wrote:


 I don't see how this setting is at all relevant to the problem at hand.


Yes, sorry, "sourcePaths" and "importPaths" were the fields I was 
thinking about. Although I think "mainSourceFile" may be needed as well, 
otherwise dub will look for "app.d".

-- 
/Jacob Carlborg
Feb 08 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 8 February 2015 at 10:39:22 UTC, Jacob Carlborg wrote:

 On 2015-02-07 20:17, Vladimir Panteleev wrote:


 I don't see how this setting is at all relevant to the problem 
 at hand.


 Yes, sorry, "sourcePaths" and "importPaths" were the fields I 
 was thinking about. Although I think "mainSourceFile" may be 
 needed as well, otherwise dub will look for "app.d".


Neither of these will help in resolving the module path and 
filesystem path inconsistency, while maintaining compatibility 
with the git submodule approach.
Feb 08 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-02-08 12:06, Vladimir Panteleev wrote:


 Neither of these will help in resolving the module path and filesystem
 path inconsistency, while maintaining compatibility with the git
 submodule approach.


I'm not exactly sure what you're looking for but with these fields you 
can configure the source directory to be something else than "source/src".

-- 
/Jacob Carlborg
Feb 08 2015
parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 8 February 2015 at 19:52:26 UTC, Jacob Carlborg wrote:

 On 2015-02-08 12:06, Vladimir Panteleev wrote:


 Neither of these will help in resolving the module path and 
 filesystem
 path inconsistency, while maintaining compatibility with the 
 git
 submodule approach.


 I'm not exactly sure what you're looking for but with these 
 fields you can configure the source directory to be something 
 else than "source/src".


I've explained it countless times in this thread. Sorry, I see no 
reason do it once again, this subthread is long enough already.
Feb 08 2015
prev sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Mon, 02 Feb 2015 10:08:58 +0000, Joseph Rushton Wakeling wrote:


 then there shouldn't be any problem with a version bump from the
 upstream application.  Whether dub actually does check for updates I'm
 not sure, but it _could_.  Whereas with git submodules, you really are
 stuck with one version and one only.


recently ;-) git got a feature "use HEAD for submodule", which removes=20
this limitation.=
Feb 02 2015
next sibling parent "Joseph Rushton Wakeling" <joseph.wakeling webdrake.net> writes:
On Monday, 2 February 2015 at 10:31:42 UTC, ketmar wrote:

 recently ;-) git got a feature "use HEAD for submodule", which 
 removes this limitation.


Nice!  If you combine that with an upstream that has 
properly-supported release branches, it's a much better situation.
Feb 02 2015
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-02-02 11:31, ketmar wrote:


 recently ;-) git got a feature "use HEAD for submodule", which removes
 this limitation.


If you refer to the 1.8.2 change that made it possible to track a branch 
in submodule. Then that doesn't make any practical difference, at least 
not in my experience.

-- 
/Jacob Carlborg
Feb 04 2015
parent "Dicebot" <public dicebot.lv> writes:
On Wednesday, 4 February 2015 at 19:07:09 UTC, Jacob Carlborg 
wrote:

 On 2015-02-02 11:31, ketmar wrote:


 recently ;-) git got a feature "use HEAD for submodule", which 
 removes
 this limitation.


 If you refer to the 1.8.2 change that made it possible to track 
 a branch in submodule. Then that doesn't make any practical 
 difference, at least not in my experience.


It adds tiny bit of convenience because it becomes possible to 
update all submodules to HEAD of their tracking branch in one go 
with trivial `git submodule update --remote`. But yes, it doesn't 
change anything in how git stores submodules in history.
Feb 04 2015
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-02-02 09:58, Joseph Rushton Wakeling via Digitalmars-d wrote:


 Scenario: a dependency has a security hole that gets patched.  If the
 dub package is updated, all applications using that dub package will
 automatically have that update available next time they are built.


That's the worst kind of behavior for security reasons. It's vital that 
you can reproduce a build any point in time. For example, building an 
application now and six months later should result in the exact same 
binary if the code of the application has not changed.

-- 
/Jacob Carlborg
Feb 04 2015
parent reply Mike Parker <aldacron gmail.com> writes:
On 2/5/2015 4:02 AM, Jacob Carlborg wrote:

 On 2015-02-02 09:58, Joseph Rushton Wakeling via Digitalmars-d wrote:


 Scenario: a dependency has a security hole that gets patched.  If the
 dub package is updated, all applications using that dub package will
 automatically have that update available next time they are built.


 That's the worst kind of behavior for security reasons. It's vital that
 you can reproduce a build any point in time. For example, building an
 application now and six months later should result in the exact same
 binary if the code of the application has not changed.


Then you specify a specific version of the library as a dependency, 
rather than a version range.
Feb 04 2015
next sibling parent =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 04.02.2015 um 23:00 schrieb Mike Parker:

 On 2/5/2015 4:02 AM, Jacob Carlborg wrote:

 On 2015-02-02 09:58, Joseph Rushton Wakeling via Digitalmars-d wrote:


 Scenario: a dependency has a security hole that gets patched.  If the
 dub package is updated, all applications using that dub package will
 automatically have that update available next time they are built.


 That's the worst kind of behavior for security reasons. It's vital that
 you can reproduce a build any point in time. For example, building an
 application now and six months later should result in the exact same
 binary if the code of the application has not changed.


 Then you specify a specific version of the library as a dependency,
 rather than a version range.


Or you commit the dub.selections.json to the repository. The good thing 
will be that DUB will still issue a message when an upstream library has 
an update available and suggests to run "dub upgrade".
Feb 05 2015
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-02-04 23:00, Mike Parker wrote:


 Then you specify a specific version of the library as a dependency,
 rather than a version range.


No, this is not enough. The tool need to automatically track and lock 
the whole dependency graph. Example:

Project A:

"dependencies": {
   "b": "1.0.0"
}

Project B:

"dependencies": {
   "c": ">=1.0.0"
}

Even though you have locked your direct dependencies to a specific 
version doesn't mean that the dependencies have done so. Meaning, you 
can get arbitrary versions of indirect dependencies. You can start 
adding the indirect dependencies as direct dependencies but that defeats 
the point of a package manager.

I have explained this so many times in the Dub forum, why not locking 
the whole graph is a really, really bad idea.

I've experienced this myself many times with Ruby before it got Bundler 
which fixes this.

-- 
/Jacob Carlborg
Feb 07 2015
parent reply Mike Parker <aldacron gmail.com> writes:
On 2/7/2015 11:43 PM, Jacob Carlborg wrote:

 On 2015-02-04 23:00, Mike Parker wrote:


 Then you specify a specific version of the library as a dependency,
 rather than a version range.


 No, this is not enough. The tool need to automatically track and lock
 the whole dependency graph. Example:

 Project A:

 "dependencies": {
    "b": "1.0.0"
 }

 Project B:

 "dependencies": {
    "c": ">=1.0.0"
 }

 Even though you have locked your direct dependencies to a specific
 version doesn't mean that the dependencies have done so. Meaning, you
 can get arbitrary versions of indirect dependencies. You can start
 adding the indirect dependencies as direct dependencies but that defeats
 the point of a package manager.

 I have explained this so many times in the Dub forum, why not locking
 the whole graph is a really, really bad idea.

 I've experienced this myself many times with Ruby before it got Bundler
 which fixes this.


But apparently this can be done now with dub.selections.json.
Feb 07 2015
parent Jacob Carlborg <doob me.com> writes:
On 2015-02-08 01:39, Mike Parker wrote:


 But apparently this can be done now with dub.selections.json.


Yes, exactly. I just replied to someone calming updating packages 
automatically was a good idea. I also had to fight quite a lot to get 
this feature into Dub.

-- 
/Jacob Carlborg
Feb 08 2015
prev sibling next sibling parent reply "Andrej Mitrovic" <andrej.mitrovich gmail.com> writes:
On Monday, 2 February 2015 at 08:09:39 UTC, Vladimir Panteleev 
wrote:

 snip


Also:

- Dub installs everything in ~/ (home, which on Windows is an 
awful location anywho). It's a pain in the ass for browsing 
dependencies in your editor. If it's just a submodule you can 
easily view it in the source tree (e.g. just open ./submodules/).

Having it as a submodule also helps if you want to quickly try to 
edit a dependency in order to see if it fixes an issue. And if 
the fix is right you can even make a commit inside the submodule 
and push it to your fork of the dependency.

With all the aliases or zsh-tricks this can be as easy as a few 
key presses.

- By default dub emits internal stack traces. This is an insane 
amount of visual noise that I don't want to read.

- If I want to try a new version of a dependency, I have to 
change the damn .json package file instead of doing a simple 'git 
checkout ...'. What's worse, I have to wait 15-20 minutes for the 
latest tagged version of a dependency to finally show up on 
code.dlang.org.

I could use add-local, but it's broken[1].

- Shit breaks every N releases (where N is arbitrary). As if it's 
not enough that code tends to break between compiler releases, so 
do packages between DUB releases. Something that used to build 
half a year ago no longer does.

I don't recall when was the last time an RDMD or Make update has 
ever broken my projects.

- I'm not a fan of poorly tested software, and DUB falls into 
that category.

I've complained before about Git submodules being a bit broken on 
Windows, maybe this was fixed in the meantime. It works solidly 
on Linux, and I'm making the decision to transfer all my projects 
that use dependencies to using submodules instead.

The bottom line is, time spent on dealing with package management 
bugs is completely wasted developer time.

[1]: 
http://forum.rejectedsoftware.com/groups/rejectedsoftware.dub/thread/5280/
Feb 02 2015
next sibling parent reply "Mathias LANG" <geod24 gmail.com> writes:
On Monday, 2 February 2015 at 09:44:18 UTC, Andrej Mitrovic wrote:

 Also:

 - Dub installs everything in ~/ (home, which on Windows is an 
 awful location anywho). It's a pain in the ass for browsing 
 dependencies in your editor. If it's just a submodule you can 
 easily view it in the source tree (e.g. just open 
 ./submodules/).


I admit it's a pain. I use Mono-D, which takes care of that, so 
I'm not running into that problem.


 Having it as a submodule also helps if you want to quickly try 
 to edit a dependency in order to see if it fixes an issue. And 
 if the fix is right you can even make a commit inside the 
 submodule and push it to your fork of the dependency.

 With all the aliases or zsh-tricks this can be as easy as a few 
 key presses.


It's a question of workflow. I use path dependency to my fork 
when I have to patch a library in order to get my project 
working. Then I P.R. and when everything's upstream, I fix the 
dependencies. It's more involved, but led me to patch Vibe.d a 
few time / report issues, so overall it's a win.


 - By default dub emits internal stack traces. This is an insane 
 amount of visual noise that I don't want to read.


Absolutely right, and it has to be patched. Fun fact: I misstyped 
Vladimir's command in a hurry, and typed "rdmd -o- file.d" 
(instead of dmd). Gotta love the stacktrace.


 - If I want to try a new version of a dependency, I have to 
 change the damn .json package file instead of doing a simple 
 'git checkout ...'. What's worse, I have to wait 15-20 minutes 
 for the latest tagged version of a dependency to finally show 
 up on code.dlang.org.

 I could use add-local, but it's broken[1].


I was not aware of that problem. Have you tried path dependencies 
?


 - Shit breaks every N releases (where N is arbitrary). As if 
 it's not enough that code tends to break between compiler 
 releases, so do packages between DUB releases. Something that 
 used to build half a year ago no longer does.

 I don't recall when was the last time an RDMD or Make update 
 has ever broken my projects.

 - I'm not a fan of poorly tested software, and DUB falls into 
 that category.



In general, things tend to break with every compiler release. 
What we need IMO (I mentionned it in the 2015H1 thread) is a CI 
system that build every package with a list of supported compiler 
and tag them accordingly.


 The bottom line is, time spent on dealing with package 
 management bugs is completely wasted developer time.


I couldn't disagree more, but I understand how frustrating dub 
can be. I also had to adapt my workflow to it.
Feb 02 2015
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Mon, 02 Feb 2015 14:36:00 +0000, Mathias LANG wrote:


 I couldn't disagree more, but I understand how frustrating dub can be. I
 also had to adapt my workflow to it.


and sometimes it's plain unusable. it insists on batch builds, and i have=20
three or so modules which eats ~2GB of RAM each on compiling (yep, CTFE-
parsing-heavy). it works with separate compilation, but batch compilation=20
just cannot be done on my x86 box.=
Feb 02 2015
parent reply =?ISO-8859-15?Q?S=F6nke_Ludwig?= <sludwig rejectedsoftware.com> writes:
Am 02.02.2015 um 15:45 schrieb ketmar:

 On Mon, 02 Feb 2015 14:36:00 +0000, Mathias LANG wrote:


 I couldn't disagree more, but I understand how frustrating dub can be. I
 also had to adapt my workflow to it.


 and sometimes it's plain unusable. it insists on batch builds, and i have
 three or so modules which eats ~2GB of RAM each on compiling (yep, CTFE-
 parsing-heavy). it works with separate compilation, but batch compilation
 just cannot be done on my x86 box.


--build-mode=singleFile
Feb 05 2015
parent ketmar <ketmar ketmar.no-ip.org> writes:
On Thu, 05 Feb 2015 15:11:50 +0100, S=C3=B6nke Ludwig wrote:


 Am 02.02.2015 um 15:45 schrieb ketmar:

 On Mon, 02 Feb 2015 14:36:00 +0000, Mathias LANG wrote:


 I couldn't disagree more, but I understand how frustrating dub can be.
 I also had to adapt my workflow to it.


 and sometimes it's plain unusable. it insists on batch builds, and i
 have three or so modules which eats ~2GB of RAM each on compiling (yep,
 CTFE- parsing-heavy). it works with separate compilation, but batch
 compilation just cannot be done on my x86 box.


 --build-mode=3DsingleFile


thanks, it's slightly better than nothing. ;-) yet it still insisting on=20
rebuilding each module, thus triggering lenghty regenerations. even for=20
libraries, where convenient tool is able to rebuild only one module and=20
replace that single object file in library.=
Feb 05 2015
prev sibling next sibling parent "Martin Nowak" <code dawg.eu> writes:
On Monday, 2 February 2015 at 09:44:18 UTC, Andrej Mitrovic wrote:

 - Dub installs everything in ~/ (home, which on Windows is an 
 awful location anywho). It's a pain in the ass for browsing 
 dependencies in your editor. If it's just a submodule you can 
 easily view it in the source tree (e.g. just open 
 ./submodules/).


It has the benefit of not reusing code and libraries across 
multiple projects.


 - By default dub emits internal stack traces. This is an insane 
 amount of visual noise that I don't want to read.


Definitely guilty of bad exception usage.


 - If I want to try a new version of a dependency, I have to 
 change the damn .json package file instead of doing a simple 
 'git checkout ...'. What's worse, I have to wait 15-20 minutes 
 for the latest tagged version of a dependency to finally show 
 up on code.dlang.org.

 I could use add-local, but it's broken[1].


Never had a problem with add-local, but I didn't use it with sub 
packages yet.
In fact sub-packages are overused and fairly complex. 
Configurations can handle most of the tasks much simpler.


 - Shit breaks every N releases (where N is arbitrary).


It's still beta and a fast moving project, but we're stabilizing.
Feb 02 2015
prev sibling next sibling parent "ponce" <contact gam3sfrommars.fr> writes:
On Monday, 2 February 2015 at 09:44:18 UTC, Andrej Mitrovic wrote:

 What's worse, I have to wait 15-20 minutes for the latest 
 tagged version of a dependency to finally show up on 
 code.dlang.org.


To avoid that, it is important to login into code.dlang.org and 
trigger manual update, to avoid that wait.
http://p0nce.github.io/d-idioms/#Patching-a-library-available-on-the-DUB-registry

Now the update will take 1 min, tops.
Feb 03 2015
prev sibling next sibling parent Martin Nowak <code+news.digitalmars dawg.eu> writes:
On 02/02/2015 10:44 AM, Andrej Mitrovic wrote:

 What's worse, I have to wait 15-20 minutes for the latest tagged version
 of a dependency to finally show up on code.dlang.org.


Filed as https://github.com/D-Programming-Language/dub-registry/issues/92.
Please comment if you have any other ideas to improve this.
Feb 03 2015
prev sibling parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 02.02.2015 um 10:44 schrieb Andrej Mitrovic:

 On Monday, 2 February 2015 at 08:09:39 UTC, Vladimir Panteleev wrote:

 snip


 Also:

 - Dub installs everything in ~/ (home, which on Windows is an awful
 location anywho). It's a pain in the ass for browsing dependencies in
 your editor. If it's just a submodule you can easily view it in the
 source tree (e.g. just open ./submodules/).


There is a request to make this configurable and it's a rather trivial 
addition. I just don't have the time to implement all feature requests 
myself, which is basically why it is not implemented, yet.

However, you are free to instead put the packages wherever you want 
instead and use "dub add-path" (or "add-local") to register it with DUB. 
You will just have to manually fetch the sources in this case.

You are also free to use git submodules and use a path based dependency 
to force using that particular instance of a package. The only caveat is 
that this will not work for public packages because there is no straight 
forward way to handle submodules properly when GitHub et al. are involved.


 - By default dub emits internal stack traces. This is an insane amount
 of visual noise that I don't want to read.


If this is true for a certain error message, please submit a ticket (or 
better yet, a quick pull request). Stack traces are supposed to be 
emitted only when the -v or --vverbose command line switches are present.


 - If I want to try a new version of a dependency, I have to change the
 damn .json package file instead of doing a simple 'git checkout ...'.
 What's worse, I have to wait 15-20 minutes for the latest tagged version
 of a dependency to finally show up on code.dlang.org.


I don't get this. You'd usually use something like `"somedep": 
"~>1.0.0"` in your dub.json. Whenever you then run "dub upgrade", it 
will update to the latest matching version.

Regarding the update frequency of the package registry, as the package 
maintainer you can often speed this up considerably by pressing the 
manual update button on the package management page. It would be better 
to use GitHub's hooks functionality to get instant updates, but that 
requires quite some more work, as well as registering each repository 
with the registry (using OAuth).


 I could use add-local, but it's broken[1].


I didn't notice that thread, but did you try the most recent version? 
There are sub module related fixes in both 0.9.22 and on GIT master that 
could affect this.


 - Shit breaks every N releases (where N is arbitrary). As if it's not
 enough that code tends to break between compiler releases, so do
 packages between DUB releases. Something that used to build half a year
 ago no longer does.


We need more regression tests in the test suite, that's true. Going by 
the relative shortage of development resources, it would be helpful to 
get some feedback on such regressions, so that they get some extra 
priority. Right now I have barely enough time to fix issues or add 
important features, so a strict regression suite policy as for DMD 
unfortunately is just not realistic ATM.


 I don't recall when was the last time an RDMD or Make update has ever
 broken my projects.

 - I'm not a fan of poorly tested software, and DUB falls into that
 category.

 I've complained before about Git submodules being a bit broken on
 Windows, maybe this was fixed in the meantime. It works solidly on
 Linux, and I'm making the decision to transfer all my projects that use
 dependencies to using submodules instead.

 The bottom line is, time spent on dealing with package management bugs
 is completely wasted developer time.

 [1]:
 http://forum.rejectedsoftware.com/groups/rejectedsoftware.dub/thread/5280/
Feb 05 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-02-05 15:11, Sönke Ludwig wrote:


 There is a request to make this configurable and it's a rather trivial
 addition. I just don't have the time to implement all feature requests
 myself, which is basically why it is not implemented, yet.


Is the reason for putting it in the home directory to be able to share 
builds between packages?

-- 
/Jacob Carlborg
Feb 07 2015
parent =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 07.02.2015 um 15:46 schrieb Jacob Carlborg:

 On 2015-02-05 15:11, Sönke Ludwig wrote:


 There is a request to make this configurable and it's a rather trivial
 addition. I just don't have the time to implement all feature requests
 myself, which is basically why it is not implemented, yet.


 Is the reason for putting it in the home directory to be able to share
 builds between packages?


Exactly. It used to be stored in a ".dub" subfolder of each package before.
Feb 09 2015
prev sibling next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 2 February 2015 at 18:09, Vladimir Panteleev via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Monday, 2 February 2015 at 05:23:52 UTC, Daniel Murphy wrote:

 [snip]


I agree with basically everything you said.
I'll add, I have no use for (and no way to use) dub because I depend
on cross-language build configurations.
If my D project depends on a C lib, then what am I supposed to do to
make dub useful for me?

I have an issue with your proposed module management solution that I
wonder if you can address.
You suggest placing source in the root of the repo. Just... no. I will
never do that. The root directory is for readme's, licenses, maybe a
build script. It's not the place for the source, that lives in 'src'.
What's the solution to this problem?
Is it that your src directory should have symlinks to the src
directories of the submodules? Is that why you raise the symlink
support on windows issue? I haven't thought of that before, and have
no idea if it works...

Does git support relative symlinks?
Feb 02 2015
next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 2 February 2015 at 11:00:20 UTC, Manu wrote:

 What's the solution to this problem?


This is a bit of a hack, but so far my solution is to further 
split up the package's modules to subpackages:

https://github.com/CyberShadow/ae


 Is it that your src directory should have symlinks to the src
 directories of the submodules? Is that why you raise the symlink
 support on windows issue? I haven't thought of that before, and 
 have no idea if it works...


Yes. It would also allow using Deimos repositories with git.

Unfortunately, it doesn't work on Windows.


 Does git support relative symlinks?


I think so - just not on Windows.
Feb 02 2015
prev sibling next sibling parent reply "Tobias Pankrath" <tobias pankrath.net> writes:
On Monday, 2 February 2015 at 11:00:20 UTC, Manu wrote:

 On 2 February 2015 at 18:09, Vladimir Panteleev via 
 Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Monday, 2 February 2015 at 05:23:52 UTC, Daniel Murphy 
 wrote:

 [snip]


 I agree with basically everything you said.
 I'll add, I have no use for (and no way to use) dub because I 
 depend
 on cross-language build configurations.
 If my D project depends on a C lib, then what am I supposed to 
 do to
 make dub useful for me?


I don't understand, can't you just use a C build tool for the C 
lib and link to it via dub?
Feb 02 2015
next sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Mon, 02 Feb 2015 11:04:29 +0000, Tobias Pankrath wrote:


 On Monday, 2 February 2015 at 11:00:20 UTC, Manu wrote:

 On 2 February 2015 at 18:09, Vladimir Panteleev via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Monday, 2 February 2015 at 05:23:52 UTC, Daniel Murphy wrote:

 [snip]


 I agree with basically everything you said.
 I'll add, I have no use for (and no way to use) dub because I depend on
 cross-language build configurations.
 If my D project depends on a C lib, then what am I supposed to do to
 make dub useful for me?


 I don't understand, can't you just use a C build tool for the C lib and
 link to it via dub?


and then we have TWO build tools. and a script that invokes one, and then=20
another. THREE build tools. see the pattern?=
Feb 02 2015
parent reply "Tobias Pankrath" <tobias pankrath.net> writes:
On Monday, 2 February 2015 at 11:59:07 UTC, ketmar wrote:

 On Mon, 02 Feb 2015 11:04:29 +0000, Tobias Pankrath wrote:


 On Monday, 2 February 2015 at 11:00:20 UTC, Manu wrote:

 On 2 February 2015 at 18:09, Vladimir Panteleev via 
 Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Monday, 2 February 2015 at 05:23:52 UTC, Daniel Murphy 
 wrote:

 [snip]


 I agree with basically everything you said.
 I'll add, I have no use for (and no way to use) dub because I 
 depend on
 cross-language build configurations.
 If my D project depends on a C lib, then what am I supposed 
 to do to
 make dub useful for me?


 I don't understand, can't you just use a C build tool for the 
 C lib and
 link to it via dub?


 and then we have TWO build tools. and a script that invokes 
 one, and then
 another. THREE build tools. see the pattern?


Well, we wont convince every C programmer out there whose library 
one wants to use to switch to dub.
Feb 02 2015
parent ketmar <ketmar ketmar.no-ip.org> writes:
On Mon, 02 Feb 2015 13:14:57 +0000, Tobias Pankrath wrote:


 Well, we wont convince every C programmer out there whose library one
 wants to use to switch to dub.


especially when dub cannot build C code.=
Feb 02 2015
prev sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 2 February 2015 at 21:04, Tobias Pankrath via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Monday, 2 February 2015 at 11:00:20 UTC, Manu wrote:

 On 2 February 2015 at 18:09, Vladimir Panteleev via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Monday, 2 February 2015 at 05:23:52 UTC, Daniel Murphy wrote:

 [snip]



 I agree with basically everything you said.
 I'll add, I have no use for (and no way to use) dub because I depend
 on cross-language build configurations.
 If my D project depends on a C lib, then what am I supposed to do to
 make dub useful for me?


 I don't understand, can't you just use a C build tool for the C lib and link
 to it via dub?


If I have another build tool, then I already have a build tool. Why
would I want 2 build tools? Double the trouble.
Feb 02 2015
next sibling parent reply "Tobias Pankrath" <tobias pankrath.net> writes:
On Monday, 2 February 2015 at 14:17:26 UTC, Manu wrote:

 On 2 February 2015 at 21:04, Tobias Pankrath via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Monday, 2 February 2015 at 11:00:20 UTC, Manu wrote:

 On 2 February 2015 at 18:09, Vladimir Panteleev via 
 Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Monday, 2 February 2015 at 05:23:52 UTC, Daniel Murphy 
 wrote:

 [snip]



 I agree with basically everything you said.
 I'll add, I have no use for (and no way to use) dub because I 
 depend
 on cross-language build configurations.
 If my D project depends on a C lib, then what am I supposed 
 to do to
 make dub useful for me?


 I don't understand, can't you just use a C build tool for the 
 C lib and link
 to it via dub?


 If I have another build tool, then I already have a build tool. 
 Why
 would I want 2 build tools? Double the trouble.


At least I don't expect Dub to support every single language out 
there natively.
Feb 02 2015
next sibling parent "Tobias Pankrath" <tobias pankrath.net> writes:
On Monday, 2 February 2015 at 14:18:48 UTC, Tobias Pankrath wrote:

 If I have another build tool, then I already have a build 
 tool. Why
 would I want 2 build tools? Double the trouble.


 At least I don't expect Dub to support every single language 
 out there natively.



 If my D project depends on a C lib, then what am I supposed to 
 do to
 make dub useful for me?


And deliberately mixing two programming languages in one project 
is another and (far more niche) problem than "depending on a 
c-lib". Many of my projects depend on a C lib and it works.
Feb 02 2015
prev sibling next sibling parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 3 February 2015 at 00:18, Tobias Pankrath via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Monday, 2 February 2015 at 14:17:26 UTC, Manu wrote:

 On 2 February 2015 at 21:04, Tobias Pankrath via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Monday, 2 February 2015 at 11:00:20 UTC, Manu wrote:

 On 2 February 2015 at 18:09, Vladimir Panteleev via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Monday, 2 February 2015 at 05:23:52 UTC, Daniel Murphy wrote:

 [snip]




 I agree with basically everything you said.
 I'll add, I have no use for (and no way to use) dub because I depend
 on cross-language build configurations.
 If my D project depends on a C lib, then what am I supposed to do to
 make dub useful for me?


 I don't understand, can't you just use a C build tool for the C lib and
 link
 to it via dub?



 If I have another build tool, then I already have a build tool. Why
 would I want 2 build tools? Double the trouble.



 At least I don't expect Dub to support every single language out there
 natively.


Right. Therefore, dub is no use to me.
There are other great cross-language build tools that support D, and
more target environments than dub, which I use instead.
Feb 02 2015
prev sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Mon, 02 Feb 2015 14:18:47 +0000, Tobias Pankrath wrote:


 At least I don't expect Dub to support every single language out there
 natively.


that's why other build tools allows to manually specify dependencies and=20
commands as a last resort. but dub can't.=
Feb 02 2015
next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 2 February 2015 at 14:59:09 UTC, ketmar wrote:

 On Mon, 02 Feb 2015 14:18:47 +0000, Tobias Pankrath wrote:


 At least I don't expect Dub to support every single language 
 out there
 natively.


 that's why other build tools allows to manually specify 
 dependencies and
 commands as a last resort. but dub can't.


Wrong on both points.
Feb 02 2015
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Mon, 02 Feb 2015 15:01:02 +0000, Dicebot wrote:


 On Monday, 2 February 2015 at 14:59:09 UTC, ketmar wrote:

 On Mon, 02 Feb 2015 14:18:47 +0000, Tobias Pankrath wrote:


 At least I don't expect Dub to support every single language out there
 natively.


 that's why other build tools allows to manually specify dependencies
 and commands as a last resort. but dub can't.


=20
 Wrong on both points.


please, can i see how dub can support building C library with gcc? nope,=20
not "just execute this very long command line and hope for the best" -- i=20
need dependency tracking, selective rebuilds and so on.=
Feb 02 2015
parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 2 February 2015 at 15:26:36 UTC, ketmar wrote:

 On Mon, 02 Feb 2015 15:01:02 +0000, Dicebot wrote:


 On Monday, 2 February 2015 at 14:59:09 UTC, ketmar wrote:

 On Mon, 02 Feb 2015 14:18:47 +0000, Tobias Pankrath wrote:


 At least I don't expect Dub to support every single language 
 out there
 natively.


 that's why other build tools allows to manually specify 
 dependencies
 and commands as a last resort. but dub can't.


 
 Wrong on both points.


 please, can i see how dub can support building C library with 
 gcc? nope,
 not "just execute this very long command line and hope for the 
 best" -- i
 need dependency tracking, selective rebuilds and so on.


It is not the same as "manually specify dependencies and 
commands".

As for dependency tracking - afaik adding dub.json to C library 
that does have any sourceFiles but simply calls existing build 
tool via" preBuildCommands" should work. Not pretty but adds all 
the metadata you have with native D package.
Feb 02 2015
parent ketmar <ketmar ketmar.no-ip.org> writes:
On Mon, 02 Feb 2015 15:29:54 +0000, Dicebot wrote:


 On Monday, 2 February 2015 at 15:26:36 UTC, ketmar wrote:

 On Mon, 02 Feb 2015 15:01:02 +0000, Dicebot wrote:


 On Monday, 2 February 2015 at 14:59:09 UTC, ketmar wrote:

 On Mon, 02 Feb 2015 14:18:47 +0000, Tobias Pankrath wrote:


 At least I don't expect Dub to support every single language out
 there natively.


 that's why other build tools allows to manually specify dependencies
 and commands as a last resort. but dub can't.


=20
 Wrong on both points.


 please, can i see how dub can support building C library with gcc?
 nope,
 not "just execute this very long command line and hope for the best" --
 i need dependency tracking, selective rebuilds and so on.


=20
 It is not the same as "manually specify dependencies and commands".


i list dependencies in my jamfile (if i can't write regexp to extact them=20
from sources, which i usually can), i specify compiler for unknown file=20
extension, i run my jam. voila, everything else is done by my jam. for=20
make it's almost similar.=
Feb 02 2015
prev sibling parent reply =?ISO-8859-15?Q?S=F6nke_Ludwig?= <sludwig rejectedsoftware.com> writes:
Am 02.02.2015 um 15:59 schrieb ketmar:

 On Mon, 02 Feb 2015 14:18:47 +0000, Tobias Pankrath wrote:


 At least I don't expect Dub to support every single language out there
 natively.


 that's why other build tools allows to manually specify dependencies and
 commands as a last resort. but dub can't.


Such as "preBuildCommands" [1]?

[1]: http://code.dlang.org/package-format#build-settings
Feb 05 2015
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Thu, 05 Feb 2015 15:51:23 +0100, S=C3=B6nke Ludwig wrote:


 Am 02.02.2015 um 15:59 schrieb ketmar:

 On Mon, 02 Feb 2015 14:18:47 +0000, Tobias Pankrath wrote:


 At least I don't expect Dub to support every single language out there
 natively.


 that's why other build tools allows to manually specify dependencies
 and commands as a last resort. but dub can't.


 Such as "preBuildCommands" [1]?
=20
 [1]: http://code.dlang.org/package-format#build-settings


nope. such as "'a' depends of 'b', 'b' depends of c, here are commands to=20
generate 'a' and 'b', don't call that commands if it's not necessary".=20
"...always before the project is built" is not what i excepting from=20
decent build tool.=
Feb 05 2015
parent reply =?ISO-8859-15?Q?S=F6nke_Ludwig?= <sludwig rejectedsoftware.com> writes:
Am 05.02.2015 um 15:56 schrieb ketmar:

 On Thu, 05 Feb 2015 15:51:23 +0100, Sönke Ludwig wrote:


 Am 02.02.2015 um 15:59 schrieb ketmar:

 On Mon, 02 Feb 2015 14:18:47 +0000, Tobias Pankrath wrote:


 At least I don't expect Dub to support every single language out there
 natively.


 that's why other build tools allows to manually specify dependencies
 and commands as a last resort. but dub can't.


 Such as "preBuildCommands" [1]?

 [1]: http://code.dlang.org/package-format#build-settings


 nope. such as "'a' depends of 'b', 'b' depends of c, here are commands to
 generate 'a' and 'b', don't call that commands if it's not necessary".
 "...always before the project is built" is not what i excepting from
 decent build tool.


Okay, so '"preBuildCommands": ["cd something && make"]' (or some other 
generic build tool instead of make)

The approach taken for DUB is to put as much knowledge of the target 
problem into the build tool as possible, so that the amount of 
work/knowledge required by the developer is minimal (as long as problem 
is within the target domain). Make's approach is the opposite and 
requires the developer to spell out every detail of the build process 
for each project. Both approaches have their advantages and DUB provides 
the command functionality specifically to enable bridging this gap.

Apart from that, directly supporting C/C++ builds is something that 
should be implemented, too, but that requires a lot of additional work.
Feb 05 2015
next sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
 The approach taken for DUB is to put as much knowledge of the 
 target problem into the build tool as possible, so that the 
 amount of work/knowledge required by the developer is minimal 
 (as long as problem is within the target domain). Make's 
 approach is the opposite and requires the developer to spell 
 out every detail of the build process for each project. Both 
 approaches have their advantages and DUB provides the command 
 functionality specifically to enable bridging this gap.


The approach is a good one. There's just a lack of flexibility. I 
know it's a lot of work, I'm definitely willing to making it 
happen.

Atila
Feb 05 2015
next sibling parent "Mathias LANG" <geod24 gmail.com> writes:
On Thursday, 5 February 2015 at 15:15:10 UTC, Atila Neves wrote:

 The approach taken for DUB is to put as much knowledge of the 
 target problem into the build tool as possible, so that the 
 amount of work/knowledge required by the developer is minimal 
 (as long as problem is within the target domain). Make's 
 approach is the opposite and requires the developer to spell 
 out every detail of the build process for each project. Both 
 approaches have their advantages and DUB provides the command 
 functionality specifically to enable bridging this gap.


 The approach is a good one. There's just a lack of flexibility. 
 I know it's a lot of work, I'm definitely willing to making it 
 happen.

 Atila


I really hope more people will get on board with with the 
development of dub, instead of everyone rolling it's own solution.
The contributor graph, while non-informative in most project, 
shows a tendency:
https://github.com/D-Programming-Language/dub/graphs/contributors
Feb 05 2015
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/5/15 7:15 AM, Atila Neves wrote:

 The approach taken for DUB is to put as much knowledge of the target
 problem into the build tool as possible, so that the amount of
 work/knowledge required by the developer is minimal (as long as
 problem is within the target domain). Make's approach is the opposite
 and requires the developer to spell out every detail of the build
 process for each project. Both approaches have their advantages and
 DUB provides the command functionality specifically to enable bridging
 this gap.


 The approach is a good one. There's just a lack of flexibility. I know
 it's a lot of work, I'm definitely willing to making it happen.


That's fantastic. -- Andrei
Feb 05 2015
prev sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Thu, 05 Feb 2015 16:12:51 +0100, S=C3=B6nke Ludwig wrote:


 Such as "preBuildCommands" [1]?

 [1]: http://code.dlang.org/package-format#build-settings


 nope. such as "'a' depends of 'b', 'b' depends of c, here are commands
 to generate 'a' and 'b', don't call that commands if it's not
 necessary". "...always before the project is built" is not what i
 excepting from decent build tool.


 Okay, so '"preBuildCommands": ["cd something && make"]' (or some other
 generic build tool instead of make)


so i'll use "make" for everything else too. "make" can be wordy, but at=20
least i'll have only one build tool to care about, not two.

there is nothing wrong in making easy things easy ;-), but dub has no way=20
to make hard things possible. i can't teach it new tricks, and i=20
definitely don't want "build systems explosion" in my projects. and dub=20
can't be easily used like "pkg-config", so even it's package management=20
part is not good for non-dub people.=
Feb 05 2015
parent reply =?ISO-8859-15?Q?S=F6nke_Ludwig?= <sludwig rejectedsoftware.com> writes:
Am 05.02.2015 um 16:35 schrieb ketmar:

 On Thu, 05 Feb 2015 16:12:51 +0100, Sönke Ludwig wrote:


 Such as "preBuildCommands" [1]?

 [1]: http://code.dlang.org/package-format#build-settings


 nope. such as "'a' depends of 'b', 'b' depends of c, here are commands
 to generate 'a' and 'b', don't call that commands if it's not
 necessary". "...always before the project is built" is not what i
 excepting from decent build tool.


 Okay, so '"preBuildCommands": ["cd something && make"]' (or some other
 generic build tool instead of make)


 so i'll use "make" for everything else too. "make" can be wordy, but at
 least i'll have only one build tool to care about, not two.


If you just want to have a build tool, that's fine (if you want to do 
all the dependency tracking and cross platform/compiler compatibility 
stuff by hand). But the main point is that you can seamlessly use 
foreign packages, which is not really possible with make (well, of 
course it is...).


 there is nothing wrong in making easy things easy ;-), but dub has no way
 to make hard things possible. i can't teach it new tricks, and i
 definitely don't want "build systems explosion" in my projects. and dub
 can't be easily used like "pkg-config", so even it's package management
 part is not good for non-dub people.


It will gain C/C++ build support, so that particular issue should at 
that point be a thing of the past. The "preBuildCommands" feature *is* a 
way to make hard things possible. It may not hit the sweet spot in your 
particular case, because it requires another tool to achieve the job, 
but it's definitely doable.

The development approach has been to try to get those features in first 
that enable the most common use cases. Certain things will always 
require the use of external tools/scripts (maybe some kind of embedded 
scripting functionality as an alternative to the command feature would 
also be an option), but it should be no problem to push the boundary of 
what's possible within the package description format to a level where 

99% of people can be happy.
Feb 05 2015
parent ketmar <ketmar ketmar.no-ip.org> writes:
On Thu, 05 Feb 2015 18:12:56 +0100, S=C3=B6nke Ludwig wrote:


 so i'll use "make" for everything else too. "make" can be wordy, but at
 least i'll have only one build tool to care about, not two.


=20
 If you just want to have a build tool, that's fine (if you want to do
 all the dependency tracking and cross platform/compiler compatibility
 stuff by hand). But the main point is that you can seamlessly use
 foreign packages, which is not really possible with make (well, of
 course it is...).


so strip out all the building duties from dub, and let it be just a=20
package manager. and then include dub package manager in D distribution=20
and teach rdmd how to use it. bingo! now D has a package manager, a=20
simple build tool, and a way to use it's package manager in other build=20
tools.


 there is nothing wrong in making easy things easy ;-), but dub has no
 way to make hard things possible. i can't teach it new tricks, and i
 definitely don't want "build systems explosion" in my projects. and dub
 can't be easily used like "pkg-config", so even it's package management
 part is not good for non-dub people.


 It will gain C/C++ build support, so that particular issue should at
 that point be a thing of the past.


that is not what dub needs, i believe. not another hard-coded support for=20
some language, but a mechanics to add support for *any* language and tool=20
without hard-coding it.


 The "preBuildCommands" feature *is* a way to make hard things possible.


it's in no way making possible building the things without resorting to=20
another build tool. so it's twice as much pain for project author: now he=20
has to learn both tools instead of only one.


 It may not hit the sweet spot in your
 particular case, because it requires another tool to achieve the job,
 but it's definitely doable.


everything dub does is doable with shell scripts too. hm. maybe dub is=20
wrongly designed from the start, and it should be just a set of shell=20
scripts for various tasks...


 The development approach has been to try to get those features in first
 that enable the most common use cases.=20


most of the time simply invoking rdmd with corresponding arguments is=20
enough. augmenting rdmd with package management is cool. having another=20
rdmd is not cool.


 but it should be no problem to push the boundary of
 what's possible within the package description format to a level where
  >99% of people can be happy.


...or simply decouple package descriptions from package build=20
instructions, and stop using dub for package building. the ONLY thing dub=20
has to know is "execute this command to build the package". that's all.=20
ah, and there must be a way to ask dub about environment, package list=20
and so on.=
Feb 06 2015
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-02-02 15:17, Manu via Digitalmars-d wrote:


 If I have another build tool, then I already have a build tool. Why
 would I want 2 build tools? Double the trouble.


In my experience most build tools are to complicated, or rather, it's 
too complicated for simple projects. Especially if they don't directly 
support the target language. A build script for an executable should be 
as simple as:

target "foo"

That would track all dependencies of the "foo.d" file and build that.

-- 
/Jacob Carlborg
Feb 04 2015
prev sibling next sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
Unless things have changed significantly or I remember 
incorrectly, vibe.d depends on C code (libevent), which gets 
built by dub.

Atila

On Monday, 2 February 2015 at 11:00:20 UTC, Manu wrote:

 On 2 February 2015 at 18:09, Vladimir Panteleev via 
 Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Monday, 2 February 2015 at 05:23:52 UTC, Daniel Murphy 
 wrote:

 [snip]


 I agree with basically everything you said.
 I'll add, I have no use for (and no way to use) dub because I 
 depend
 on cross-language build configurations.
 If my D project depends on a C lib, then what am I supposed to 
 do to
 make dub useful for me?

 I have an issue with your proposed module management solution 
 that I
 wonder if you can address.
 You suggest placing source in the root of the repo. Just... no. 
 I will
 never do that. The root directory is for readme's, licenses, 
 maybe a
 build script. It's not the place for the source, that lives in 
 'src'.
 What's the solution to this problem?
 Is it that your src directory should have symlinks to the src
 directories of the submodules? Is that why you raise the symlink
 support on windows issue? I haven't thought of that before, and 
 have
 no idea if it works...

 Does git support relative symlinks?
Feb 02 2015
next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 2 February 2015 at 12:54:02 UTC, Atila Neves wrote:

 Unless things have changed significantly or I remember 
 incorrectly, vibe.d depends on C code (libevent), which gets 
 built by dub.


At least for Windows, Vibe includes pre-built libevent binaries 
(DLL and import library) in the git repository.
Feb 02 2015
prev sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
On Mon, 02 Feb 2015 12:54:01 +0000, Atila Neves wrote:


 Unless things have changed significantly or I remember incorrectly,
 vibe.d depends on C code (libevent), which gets built by dub.


no, it's not build by dub. at least for GNU/Linux it expects to find it=20
already installed.=
Feb 02 2015
prev sibling next sibling parent "Jonathan Marler" <johnnymarler gmail.com> writes:
On Monday, 2 February 2015 at 11:00:20 UTC, Manu wrote:

 I have an issue with your proposed module management solution 
 that I
 wonder if you can address.
 You suggest placing source in the root of the repo. Just... no. 
 I will
 never do that. The root directory is for readme's, licenses, 
 maybe a
 build script. It's not the place for the source, that lives in 
 'src'.
 What's the solution to this problem?
 Is it that your src directory should have symlinks to the src
 directories of the submodules? Is that why you raise the symlink
 support on windows issue? I haven't thought of that before, and 
 have
 no idea if it works...

 Does git support relative symlinks?


If you're going to start using submodules, how about putting your 
source innto it's own submodule?

fruit ->
    README
    src -> (git submodule "fruit_src"?)
        apple.d
        banana.d

Just a thought.  If git supported cloning a partial repository 
you could do this without a sub module as well.  What do you 
think?  Seems a little odd to me but maybe it could work.
Feb 02 2015
prev sibling next sibling parent Martin Nowak <code+news.digitalmars dawg.eu> writes:
On 02/02/2015 12:00 PM, Manu via Digitalmars-d wrote:

 If my D project depends on a C lib, then what am I supposed to do to
 make dub useful for me?


This is a simple problem to solve.
All package tools support a way to build "native" extensions, mostly by 
scripting the builds.
You can already add preBuildCommands and use whatever you like, e.g. a D 
script to build your C library.
It wouldn't be too hard to compile just a few C files, but if you 
already need this, chances are you need more flexibility anyhow.
Feb 03 2015
prev sibling parent =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 02.02.2015 um 12:00 schrieb Manu via Digitalmars-d:

 On 2 February 2015 at 18:09, Vladimir Panteleev via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Monday, 2 February 2015 at 05:23:52 UTC, Daniel Murphy wrote:

 [snip]


 I agree with basically everything you said.
 I'll add, I have no use for (and no way to use) dub because I depend
 on cross-language build configurations.
 If my D project depends on a C lib, then what am I supposed to do to
 make dub useful for me?


I agree that this should have the highest priority on the feature list, 
especially considering all the effort that goes into improving the C++ 
interface. My plan was to get something useful for a pure D ecosystem 
first and then explore the more advanced use cases.
Feb 05 2015
prev sibling next sibling parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, 2015-02-02 at 08:09 +0000, Vladimir Panteleev via Digitalmars-d wrote:

 


[…]

 2. git
 
 I've found that git's submodule feature is an excellent way to 
 manage dependencies across D libraries, and fits really well with 
 D's package system. For clarity to readers not too familiar with 
 Git, I'll explain by example:
 
 […]


Go allows for Git, Mercurial and Bazaar. This is a good idea since it 
obviates the "but you are not using the DVCS I want to use" problem.

The problem with Git is versioning. Gradle, Maven, Herd (Ceylon 
package repository) handle this easily. Go has a real problem with 
this and Gustavo created a versioning strategy that works well. cf. 
http://labix.org/gopkg.in

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Feb 02 2015
prev sibling next sibling parent reply "Mathias LANG" <geod24 gmail.com> writes:
On Monday, 2 February 2015 at 08:09:39 UTC, Vladimir Panteleev 
wrote:

 In contrast, Dub's default modus operandi is to blindly send to 
 the compiler all *.d files found in the "src" folder, whether 
 they're actually used or not. Not only can this be slower if 
 not all modules are always used, but it also won't work if the 
 source code contains multiple entry points, forcing you to 
 write complicated configuration files (i.e. do the computer's 
 work for it).


To be more specific, dub won't let you compile a project with 
multiple definition of a function. How is that a liability ?
Is rdmd able to build static and dynamic library ? If so, does it 
ignore files that are not imported anywhere ?


 1b. rdmd and D's search path

 rdmd does not have any additional parameters to set up for 
 where it needs to look for source files, because it relies on 
 the compiler's search mechanism. Thus, if you can build your 
 program with rdmd, "dmd -o- program" will succeed, and usually 
 vice versa.

 In contrast, Dub builds its own search path using its JSON 
 configuration files, and has no equivalent of "dmd -o-".


I don't see any downside here.


 There is no simple way to syntax-check just one file in a 
 project when using Dub. I rely on this a lot in my workflow - I 
 configured a syntax-check key in my editor, which I use almost 
 as often as I save. A syntax check (dmd -o-) is much faster 
 than a full build, as it skips parsing other parts of the 
 project, code generation, and linking.


What's your editor ? Mine complains on syntax error.
If you're using vi/emacs, then placing your editor in source/ (or 
starting dmd here) would do the trick.


 2d. Git vs. Dub

 Unfortunately, the above-described approach is not compatible 
 with Dub:

 - Dub packages are generally expected to have their source code 
 in a "src" subdirectory, although you can set the source 
 directory to "." in the configuration file.

 - When cloning repositories, dub does not preserve the 
 repository's directory name (so e.g. fruit will be cloned to 
 ~/.dub/fruit-1.0.0/).

 Somebody has created a Dub package for my library (excluding 
 certain packages, due to point 1a above), and the other day 
 someone complained that it doesn't work with Dscanner, because 
 of the above issue - the module path "ae.dir.module" does not 
 correspond to the filesystem path "ae-1.0.1/dir/module.d".


git clone http://github.com/You/repo otherName

breaks that workflow equally. Relying on the name under which 
your repo was cloned doesn't seem that robust.


 So, in order to start using Dub, I'd need to:

 - restructure the directory layout of my library (breaking 
 change)
 - update all projects which use this library to use Dub instead
 - give up quick syntax checking
 - give up commit-granularity versioning
 - begin maintaining JSON configuration files
 - begin versioning libraries by hand
 - install Dub on all my computers, servers, and virtual machines

 No thanks.


- But then your library is self contained and won't break if 
someone has another workflow that makes him clone your library 
under a different name.
- That's the point.
- Addressed;
- Semver allows you to do much more, as mentionned before;
- It isn't much of a maintainance. You rarely edit dub.json once 
the base is set.
- dub rely on git tags for versioning. If you want to do *real* 
versioning (and not just "most up to date tag"), you'll still 
have to play with branches and submodules.
- Only your dev stations.


 Change my view.


Hope I helped :)
Feb 02 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 2 February 2015 at 13:25:57 UTC, Mathias LANG wrote:

 To be more specific, dub won't let you compile a project with 
 multiple definition of a function. How is that a liability ?


You can't have more than one main() function.

Some packages contain more than one entry point (programs / 
executables).

In my case, my library allows selecting an entry point based on 
the nature of the program (console/GUI) and the platform. The 
program only imports one of these. Vibe does something similar, 
but it has only one entry point.


 Is rdmd able to build static and dynamic library ?


Yes.


 If so, does it ignore files that are not imported anywhere ?


Yes.


 I don't see any downside here.


The downside is dmd -o- doesn't work.


 What's your editor ? Mine complains on syntax error.
 If you're using vi/emacs, then placing your editor in source/ 
 (or starting dmd here) would do the trick.


"syntax check" was not meant literally. Can your editor 
instantiate imported templates, too?

FYI, my editor's D syntax highlighting is probably more extensive 
than any others':

https://github.com/colorer/Colorer-schemes/blob/master/hrc/hrc/base/d.hrc


 git clone http://github.com/You/repo otherName

 breaks that workflow equally. Relying on the name under which 
 your repo was cloned doesn't seem that robust.


The distinction is that you have to go out of your way to break 
it. Dub not just breaks it by default, but there is no simple way 
to work around it, either.


 - But then your library is self contained and won't break if 
 someone has another workflow that makes him clone your library 
 under a different name.


So Dub protects people from shooting themselves in the foot by 
cutting their feet off? How nice.


 - dub rely on git tags for versioning. If you want to do *real* 
 versioning (and not just "most up to date tag"), you'll still 
 have to play with branches and submodules.


Yay, more buttons to press! When I could actually be getting 
things done instead.


 Hope I helped :)


Sorry, not even close.
Feb 02 2015
next sibling parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Monday, 2 February 2015 at 13:42:19 UTC, Vladimir Panteleev 
wrote:

 On Monday, 2 February 2015 at 13:25:57 UTC, Mathias LANG wrote:

 To be more specific, dub won't let you compile a project with 
 multiple definition of a function. How is that a liability ?


 You can't have more than one main() function.

 Some packages contain more than one entry point (programs / 
 executables).

 In my case, my library allows selecting an entry point based on 
 the nature of the program (console/GUI) and the platform. The 
 program only imports one of these. Vibe does something similar, 
 but it has only one entry point.


You can do this with dub. You selectively include or exclude 
files/directories on a per-configuration basis.

Overall - while you do have some valid complaints - you don't 
seems to know dub particularly well. Maybe you do, but it's not 
coming across.
Feb 02 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 2 February 2015 at 14:06:46 UTC, John Colvin wrote:

 You can do this with dub. You selectively include or exclude 
 files/directories on a per-configuration basis.


I have addressed this in my initial post.


 Overall - while you do have some valid complaints - you don't 
 seems to know dub particularly well. Maybe you do, but it's not 
 coming across.


I don't understand what point you're trying to say here. The 
thread starts with the line:


 I don't use Dub




Andrej and I seem to have a very similar workflow, as the 
additional experience he shared would affect me as well if I 
tried to switch to it.
Feb 02 2015
parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Monday, 2 February 2015 at 14:09:55 UTC, Vladimir Panteleev 
wrote:

 On Monday, 2 February 2015 at 14:06:46 UTC, John Colvin wrote:

 You can do this with dub. You selectively include or exclude 
 files/directories on a per-configuration basis.


 I have addressed this in my initial post.


Not really. You say that it amounts to doing the computers work 
for it. I don't understand how the computer is supposed to know, 
in the general case, which files are relevant for which 
configurations. You have to tell it somehow. Dub has various 
features that can be leveraged to achieve this goal.


 Overall - while you do have some valid complaints - you don't 
 seems to know dub particularly well. Maybe you do, but it's 
 not coming across.


 I don't understand what point you're trying to say here. The 
 thread starts with the line:


 I don't use Dub




 Andrej and I seem to have a very similar workflow, as the 
 additional experience he shared would affect me as well if I 
 tried to switch to it.


It seems there are 4 genuine problems that have been mentioned:

1) Dependency versioning at the git commit level.
2) Placing dependencies out-of-tree.
3) Cross-language builds.
4) Bugs.

Solutions:
1) Support for this could be added to dub. Alternatively dub 
packages could be based on a recursive clone, so if you really 
need submodules then you get them, but your package still works 
with dub.

2) --cache    I don't use it, but it seems like it would do what 
you want. Out-of-tree does have advantages though. Auto-created 
local symlinks to the global package cache would probably be 
best-of-both-worlds.

3) Genuinely difficult. Probably outside of the scope of dub.

4) The situations is better than it used to be. Note that dub is 
still pre-1.0
Feb 02 2015
prev sibling parent reply "Mathias LANG" <geod24 gmail.com> writes:
On Monday, 2 February 2015 at 13:42:19 UTC, Vladimir Panteleev 
wrote:

 On Monday, 2 February 2015 at 13:25:57 UTC, Mathias LANG wrote:

 To be more specific, dub won't let you compile a project with 
 multiple definition of a function. How is that a liability ?


 You can't have more than one main() function.

 Some packages contain more than one entry point (programs / 
 executables).

 In my case, my library allows selecting an entry point based on 
 the nature of the program (console/GUI) and the platform. The 
 program only imports one of these. Vibe does something similar, 
 but it has only one entry point.


You can do that using configuration sections:
http://code.dlang.org/package-format#configurations


 Is rdmd able to build static and dynamic library ?


 Yes.


 If so, does it ignore files that are not imported anywhere ?


 Yes.


I don't see how this is a feature. I want all my file parsed and 
put inside the archive.


 I don't see any downside here.


 The downside is dmd -o- doesn't work.


It works when you're in source. No difference.


 What's your editor ? Mine complains on syntax error.
 If you're using vi/emacs, then placing your editor in source/ 
 (or starting dmd here) would do the trick.


 "syntax check" was not meant literally. Can your editor 
 instantiate imported templates, too?


I'd like it too, but not ATM :(
Is the working directory option working for you ?


 git clone http://github.com/You/repo otherName

 breaks that workflow equally. Relying on the name under which 
 your repo was cloned doesn't seem that robust.


 The distinction is that you have to go out of your way to break 
 it. Dub not just breaks it by default, but there is no simple 
 way to work around it, either.


If you go out of dub's way, you might break it. The same happen 
when someone goes out of your workflow. Is running dmd -o- from 
./project/source instead of ./project/ impossible with your 
setting / IDE ?


 - dub rely on git tags for versioning. If you want to do 
 *real* versioning (and not just "most up to date tag"), you'll 
 still have to play with branches and submodules.


 Yay, more buttons to press! When I could actually be getting 
 things done instead.


Want to track a branch: ~branchName
Want to track a specific commit: == x.x.x
want to track a bounded commit range: ~> / >= <=
Want to track an unbounded commit range: >=
Feb 02 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 2 February 2015 at 14:11:12 UTC, Mathias LANG wrote:

 You can do that using configuration sections:
 http://code.dlang.org/package-format#configurations


I addressed this in my original post.


 I don't see how this is a feature. I want all my file parsed 
 and put inside the archive.


Use Dub, then. You asked if it can be done with rdmd.


 It works when you're in source. No difference.


No. It doesn't work if your project has dependencies. Which was 
my point entirely.

We are going in circles. I don't think there's much more to be 
said.
Feb 02 2015
parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Monday, 2 February 2015 at 14:25:35 UTC, Vladimir Panteleev 
wrote:

 On Monday, 2 February 2015 at 14:11:12 UTC, Mathias LANG wrote:

 You can do that using configuration sections:
 http://code.dlang.org/package-format#configurations


 I addressed this in my original post.


 I don't see how this is a feature. I want all my file parsed 
 and put inside the archive.


 Use Dub, then. You asked if it can be done with rdmd.


 It works when you're in source. No difference.


 No. It doesn't work if your project has dependencies. Which was 
 my point entirely.

 We are going in circles. I don't think there's much more to be 
 said.


a little script that calls `dub describe`, extracts the import 
paths and then calls dmd -o- appropriately would be easy to do. 
There's no fundamental blocker here.
Feb 02 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 2 February 2015 at 14:35:03 UTC, John Colvin wrote:

 a little script that calls `dub describe`, extracts the import 
 paths and then calls dmd -o- appropriately would be easy to do. 
 There's no fundamental blocker here.


As long as the source code is open, there is no fundamental 
blocker at all - only a scalar of required effort, and the 
resulting gain per effort expended. But in my case, nothing is 
blocked - it is not a question of "I want to use dub but can't", 
but "I can't use dub and don't really need to".
Feb 02 2015
parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 02.02.2015 um 16:34 schrieb Vladimir Panteleev:

 On Monday, 2 February 2015 at 14:35:03 UTC, John Colvin wrote:

 a little script that calls `dub describe`, extracts the import paths
 and then calls dmd -o- appropriately would be easy to do. There's no
 fundamental blocker here.


 As long as the source code is open, there is no fundamental blocker at
 all - only a scalar of required effort, and the resulting gain per
 effort expended. But in my case, nothing is blocked - it is not a
 question of "I want to use dub but can't", but "I can't use dub and
 don't really need to".


BTW, for what it's worth, you can also do this:
"DFLAGS=-o- dub build"

It will recognize the -o- and automatically skip the linker stage, too. 
But of course that doesn't make it automatically track dependencies.
Feb 05 2015
parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Thursday, 5 February 2015 at 15:14:53 UTC, Sönke Ludwig wrote:

 BTW, for what it's worth, you can also do this:
 "DFLAGS=-o- dub build"

 It will recognize the -o- and automatically skip the linker 
 stage, too. But of course that doesn't make it automatically 
 track dependencies.


Nice! Then all it needs is a parameter for the file to check.
Feb 05 2015
prev sibling next sibling parent reply "Dicebot" <public dicebot.lv> writes:
Quick summary of my opinion can look like this:

1) dub is a horrible build tool
2) git submodules is a horrible way for dependency management

First thing is something we can improve on in future and, 
actually, dub supports using rdmd as a build tool already.

Second thing is an inherent issue that won't change without some 
fundamental breaking changes in how hit submodules are 
implemented.

Because of that I am trying to push dub despite the fact I don't 
like lot of things about it personally.

----------------------------------------

Explaining why dub is a horrible build tool is hardly necessary - 
there are plenty of cases in this thread already.

So why git submodules don't really work?

1) Lack of version management

submodule remembers specific commit hash and that is all. As of 
recent git version one can also remember remote tracking branch 
(but it will still keep specific hash in history!)

This may not seem an issue if you have only few dependencies and 
control them all. But it scales badly once bulding the project 
needs many incompatible versions of the same dependency and 
dependency count increases in general.

2) Lack of local cache

This is really annoying when working with many project with 
similar dependencies - those will get cloned over and over again, 
taking quite some time. Can possibly be fixed with clever scripts 
that replace remotes with local repo paths but that is not 
out-of-the box solution

3) Lack of package repository

Biggest thing about dub is not dub tool itself. It is 
code.dlang.org and what it can become in future. Providing simple 
centralized way to discover new libraries and quickly try those 
in uniform matter is the way you build decentralized ecosystem 
(ironically).

With git submodules you are pretty much stick with doing GitHub 
search over and over again

4) Inherently coupled with specific DVCS

git and GitHub are most common (and officially endorsed) ways of 
version control in D world but those are not the only one. dub 
registry allows to use GitHub, GitBucket and source tarball 
packages in the same project - in simple uniform matter. It is a 
100% in-house tool that does no depend on decisions of external 
project maintainers that provides a common API for everything 
else to interoperate
Feb 02 2015
next sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
This thread has just made me decide to write a D build system, in 
D and configured in D that builds on dub and its packages intead 
of reinventing the wheel. I might ask the community for inputs on 
requirements. Maybe this will be my DConf 2015 submission.

Also, Dicebot, I agree with nearly everything you said here. It 
tends to happen that I usually either completely agree or 
completely disagree with you ;)

Atila

On Monday, 2 February 2015 at 14:41:00 UTC, Dicebot wrote:

 Quick summary of my opinion can look like this:

 1) dub is a horrible build tool
 2) git submodules is a horrible way for dependency management

 First thing is something we can improve on in future and, 
 actually, dub supports using rdmd as a build tool already.

 Second thing is an inherent issue that won't change without 
 some fundamental breaking changes in how hit submodules are 
 implemented.

 Because of that I am trying to push dub despite the fact I 
 don't like lot of things about it personally.

 ----------------------------------------

 Explaining why dub is a horrible build tool is hardly necessary 
 - there are plenty of cases in this thread already.

 So why git submodules don't really work?

 1) Lack of version management

 submodule remembers specific commit hash and that is all. As of 
 recent git version one can also remember remote tracking branch 
 (but it will still keep specific hash in history!)

 This may not seem an issue if you have only few dependencies 
 and control them all. But it scales badly once bulding the 
 project needs many incompatible versions of the same dependency 
 and dependency count increases in general.

 2) Lack of local cache

 This is really annoying when working with many project with 
 similar dependencies - those will get cloned over and over 
 again, taking quite some time. Can possibly be fixed with 
 clever scripts that replace remotes with local repo paths but 
 that is not out-of-the box solution

 3) Lack of package repository

 Biggest thing about dub is not dub tool itself. It is 
 code.dlang.org and what it can become in future. Providing 
 simple centralized way to discover new libraries and quickly 
 try those in uniform matter is the way you build decentralized 
 ecosystem (ironically).

 With git submodules you are pretty much stick with doing GitHub 
 search over and over again

 4) Inherently coupled with specific DVCS

 git and GitHub are most common (and officially endorsed) ways 
 of version control in D world but those are not the only one. 
 dub registry allows to use GitHub, GitBucket and source tarball 
 packages in the same project - in simple uniform matter. It is 
 a 100% in-house tool that does no depend on decisions of 
 external project maintainers that provides a common API for 
 everything else to interoperate
Feb 02 2015
next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 2 February 2015 at 16:26:45 UTC, Atila Neves wrote:

 This thread has just made me decide to write a D build system, 
 in D and configured in D that builds on dub and its packages 
 intead of reinventing the wheel. I might ask the community for 
 inputs on requirements. Maybe this will be my DConf 2015 
 submission.


I have recently been experimenting with meta-dlang repository (to 
aggregate exisiting ones) and using D scripts instead of 
makefiles for uniform cross-platform build experience.

That quickly has made me wanting to get something like `std.make` 
into Phobos - small utility module that would allow declaratively 
defining target tree + shell scripts similar to makefiles, as 
well as set of small nice shell wrappers (like one that 
effectively enables `set -e` for called commands). Having such 
module in standard library would make it much easier to replace 
makefiles with a quick D scripts and potentially build a robust 
build system on top.
Feb 02 2015
next sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
On Monday, 2 February 2015 at 16:42:03 UTC, Dicebot wrote:

 On Monday, 2 February 2015 at 16:26:45 UTC, Atila Neves wrote:

 This thread has just made me decide to write a D build system, 
 in D and configured in D that builds on dub and its packages 
 intead of reinventing the wheel. I might ask the community for 
 inputs on requirements. Maybe this will be my DConf 2015 
 submission.


 I have recently been experimenting with meta-dlang repository 
 (to aggregate exisiting ones) and using D scripts instead of 
 makefiles for uniform cross-platform build experience.

 That quickly has made me wanting to get something like 
 `std.make` into Phobos - small utility module that would allow 
 declaratively defining target tree + shell scripts similar to 
 makefiles, as well as set of small nice shell wrappers (like 
 one that effectively enables `set -e` for called commands). 
 Having such module in standard library would make it much 
 easier to replace makefiles with a quick D scripts and 
 potentially build a robust build system on top.


I have ideas that go beyond this, but this is useful input. I 
wonder how to proceed now about gathering actual requirements 
from D devs and seeing which ones are important.

Atila
Feb 02 2015
parent reply Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, 2015-02-02 at 16:50 +0000, Atila Neves via Digitalmars-d wrote:
[…]

 
 I have ideas that go beyond this, but this is useful input. I 
 wonder how to proceed now about gathering actual requirements 
 from D devs and seeing which ones are important.


Learn lessons from SBT vs. SCons/Gant/Gradle: SBT is a Scala build
system using Scala programs as input.

Some Scala folk got religious about Scala being the only right language
for Scala builds, and created what has become SBT. Because of the way
statically typed Scala works as a domain specific languages there are
performance and expression issues that means SBT is getting a reputation
in the very companies that should be supporting it that it is "the
enemy".

A D build system must trivially support C, C++(, and Fortran?).

Transitive dependency management needs careful consideration. I haven't
investigated whether Dub does this right, it may well do. Maven does
not, Gradle does.

Define package, artefact, dependency carefully. Go has a strong model
(even if it is wrong). Ceylon improves on Java/Scal/etc. Python got it
woefully wrong: eggs were a mess, and PyPI is not curated well enough
(the vast majority of stuff on PyPI is unusable rubbish).

Worry about platform specific packagers: MacPorts, Homebrew, Debian,
Fedora, FreeBSD, OpenBSD, etc. will (hopefully) want to package. Java is
generally a disaster for them, and Python isn't that much better. Go is
ignoring al this and creating a monoculture based on Git and Mercurial
as packages are shipped as source. I see Debian and Fedora trying to
package Go things and predict the same mess as Java and Python.

Support principally a declarative DSL, but with imperative structure
possible. A build specification is a meta-program which when executed
creates the program and hence the build process.

Have no clearly silly constraints, cf. the needs for SBT specification
lines always to be separated by blank lines.

Be convention over configuration, but allow configuration.

Use BinTray and Artifactory as well as the current Dub repository.

Do not be afraid to change the meta-data specification so that. Dub may
well be a good start, but it may need work. Gradle has changed it's
meta-data specifications many times despite being constrained by
compatibility with Maven POMs and Ivy specification.

Stand on the shoulders of giants, but do not assume they are always
right, be prepared to change stuff for the better.

Get alpha testers in early and let them drive things – albeit within the
boundaries of your vision for the system. Dub did well here if I
remember correctly and created a group of people who did and emailed
rather than just emailing. 

I better finish my essay at this point I have a London D User Group
meeting to go to :-)

http://www.meetup.com/London-D-Programmers/


-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Feb 03 2015
next sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
This is really good feedback, thanks. Would you be interested in 
discussing this further privately? I've already got some ideas 
brewing in my head.

Atila

On Tuesday, 3 February 2015 at 18:00:31 UTC, Russel Winder wrote:

 On Mon, 2015-02-02 at 16:50 +0000, Atila Neves via 
 Digitalmars-d wrote:
 […]

 
 I have ideas that go beyond this, but this is useful input. I 
 wonder how to proceed now about gathering actual requirements 
 from D devs and seeing which ones are important.


 Learn lessons from SBT vs. SCons/Gant/Gradle: SBT is a Scala 
 build
 system using Scala programs as input.

 Some Scala folk got religious about Scala being the only right 
 language
 for Scala builds, and created what has become SBT. Because of 
 the way
 statically typed Scala works as a domain specific languages 
 there are
 performance and expression issues that means SBT is getting a 
 reputation
 in the very companies that should be supporting it that it is 
 "the
 enemy".

 A D build system must trivially support C, C++(, and Fortran?).

 Transitive dependency management needs careful consideration. I 
 haven't
 investigated whether Dub does this right, it may well do. Maven 
 does
 not, Gradle does.

 Define package, artefact, dependency carefully. Go has a strong 
 model
 (even if it is wrong). Ceylon improves on Java/Scal/etc. Python 
 got it
 woefully wrong: eggs were a mess, and PyPI is not curated well 
 enough
 (the vast majority of stuff on PyPI is unusable rubbish).

 Worry about platform specific packagers: MacPorts, Homebrew, 
 Debian,
 Fedora, FreeBSD, OpenBSD, etc. will (hopefully) want to 
 package. Java is
 generally a disaster for them, and Python isn't that much 
 better. Go is
 ignoring al this and creating a monoculture based on Git and 
 Mercurial
 as packages are shipped as source. I see Debian and Fedora 
 trying to
 package Go things and predict the same mess as Java and Python.

 Support principally a declarative DSL, but with imperative 
 structure
 possible. A build specification is a meta-program which when 
 executed
 creates the program and hence the build process.

 Have no clearly silly constraints, cf. the needs for SBT 
 specification
 lines always to be separated by blank lines.

 Be convention over configuration, but allow configuration.

 Use BinTray and Artifactory as well as the current Dub 
 repository.

 Do not be afraid to change the meta-data specification so that. 
 Dub may
 well be a good start, but it may need work. Gradle has changed 
 it's
 meta-data specifications many times despite being constrained by
 compatibility with Maven POMs and Ivy specification.

 Stand on the shoulders of giants, but do not assume they are 
 always
 right, be prepared to change stuff for the better.

 Get alpha testers in early and let them drive things – albeit 
 within the
 boundaries of your vision for the system. Dub did well here if I
 remember correctly and created a group of people who did and 
 emailed
 rather than just emailing.

 I better finish my essay at this point I have a London D User 
 Group
 meeting to go to :-)

 http://www.meetup.com/London-D-Programmers/
Feb 03 2015
parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 2015-02-03 at 21:02 +0000, Atila Neves via Digitalmars-d wrote:

 This is really good feedback, thanks. Would you be interested in 
 discussing this further privately? I've already got some ideas 
 brewing in my head.


No problem, glad it might be helpful.

I am happy to be involved.

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Feb 04 2015
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-02-03 19:00, Russel Winder via Digitalmars-d wrote:


 Learn lessons from SBT vs. SCons/Gant/Gradle: SBT is a Scala build
 system using Scala programs as input.

 Some Scala folk got religious about Scala being the only right language
 for Scala builds, and created what has become SBT. Because of the way
 statically typed Scala works as a domain specific languages there are
 performance and expression issues that means SBT is getting a reputation
 in the very companies that should be supporting it that it is "the
 enemy".


It's kind of the same in the D community. Before Dub existed I started 
to work on both a package manager (Orbit) and a build tool (never 
officially announced). I used Ruby as the description format for package 
files a build scripts. Ruby is a language which allows you to create 
DSL's both with a declarative syntax and still support imperative code. 
Example:

target :foo

if System.osx
   target :foobar
end

Of course that was completely reject since it was Ruby and not D. 
Although, for some reason Make is acceptable in the D community. It's 
basically only D or a markup language that will be accepted.

-- 
/Jacob Carlborg
Feb 08 2015
next sibling parent reply Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 2015-02-08 at 12:00 +0100, Jacob Carlborg via Digitalmars-d wrote:

=20


[=E2=80=A6]

 It's kind of the same in the D community. Before Dub existed I=20
 started to work on both a package manager (Orbit) and a build tool=20
 (never officially announced). I used Ruby as the description format=20
 for package
 files a build scripts. Ruby is a language which allows you to create=20
 DSL's both with a declarative syntax and still support imperative=20
 code. Example:
=20
 target :foo
=20
 if System.osx
    target :foobar
 end
=20
 Of course that was completely reject since it was Ruby and not D.



Well clearly it should be rejected, you have to s/Ruby/Python/ ;-)

I worked on Rant for a while because Rake is not up to building C and=20
C++ systems due to lack of abstractions. As far as I can tell Rake=20
remain the de facto standard tool in the Ruby community and absolutely=20
nowhere else.

The demise of the Rant project and my shift to SCons as I needed a C++=20
and LaTeX build framework meant I dropped Ruby and moved to Python.=20
Well there are other reasons I feel Python is better than Ruby, but=20
whilst some are objective, many are purely subjective.


 Although, for some reason Make is acceptable in the D community.


Possibly because it pre-dates C++?


 It's basically only D or a markup language that will be accepted.


SBT has, or rather had, many things going for it, but it's problems=20
will either get fixed or it will face a (n admittedly slow due to the=20
energy of its adherents) slow decline into obscurity.

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Feb 08 2015
next sibling parent FG <home fgda.pl> writes:
On 2015-02-08 at 14:59, Russel Winder via Digitalmars-d wrote:

 Well clearly it should be rejected, you have to s/Ruby/Python/ ;-)


LOL


 As far as I can tell Rake remain the de facto standard tool
 in the Ruby community and absolutely nowhere else.


That's by design. A rake gathers hay towards the person using it,
and makes the reverse, spreading, very inconvenient. :)
Feb 08 2015
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-02-08 14:59, Russel Winder via Digitalmars-d wrote:


 I worked on Rant for a while because Rake is not up to building C and
 C++ systems due to lack of abstractions. As far as I can tell Rake
 remain the de facto standard tool in the Ruby community and absolutely
 nowhere else.


Since Rake will give you the full power of Ruby I assume you mean some 
high level helper constructs?

-- 
/Jacob Carlborg
Feb 08 2015
parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 2015-02-08 at 20:54 +0100, Jacob Carlborg via Digitalmars-d
wrote:
[=E2=80=A6]

=20
 Since Rake will give you the full power of Ruby I assume you mean some=


=20

 high level helper constructs?


The Rake community were not really interested in supporting C and C++
builds, let alone LaTeX, with abstractions over the base infrastructure,
and (I can't remember the details) there were some problems in the Rake
ADG processing that meant it was hard to create good abstractions for C
and C++. Rant was a Rake for processing these higher level abstractions.

Rake is basically 1977 Make without the knowledge of C, C++, Yacc, Lex,
etc., where Rant was more like 2007 GNU Make.

Yes Rake can be used for C and C++ builds, but you end up having to
write the abstractions, and the community (at least back then) was not
interested in supporting them in the library because it wasn't Ruby
oriented. Ho hummm=E2=80=A6

It was a pity, I liked Rant, and wrote a whole LaTeX processing
infrastructure. Of course, fortunately, as the Rant project was grinding
to a halt I discovered SCons already had all the LaTeX stuff I had been
writing.

It would have been nice if Rake had had more take up in non-Ruby areas,
but it didn't and it is almost certainly too late now.

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Feb 09 2015
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/8/15 3:00 AM, Jacob Carlborg wrote:

 On 2015-02-03 19:00, Russel Winder via Digitalmars-d wrote:


 Learn lessons from SBT vs. SCons/Gant/Gradle: SBT is a Scala build
 system using Scala programs as input.

 Some Scala folk got religious about Scala being the only right language
 for Scala builds, and created what has become SBT. Because of the way
 statically typed Scala works as a domain specific languages there are
 performance and expression issues that means SBT is getting a reputation
 in the very companies that should be supporting it that it is "the
 enemy".


 It's kind of the same in the D community. Before Dub existed I started
 to work on both a package manager (Orbit) and a build tool (never
 officially announced). I used Ruby as the description format for package
 files a build scripts. Ruby is a language which allows you to create
 DSL's both with a declarative syntax and still support imperative code.
 Example:

 target :foo

 if System.osx
    target :foobar
 end

 Of course that was completely reject since it was Ruby and not D.
 Although, for some reason Make is acceptable in the D community. It's
 basically only D or a markup language that will be accepted.


Ehm. This part sounds unnecessarily a bit political - NIH syndrome, 
closemindedness of the D folks toward using anything else but D and 
make, and such.

I do remember one action I took "against" Ruby - replacing a 109 line 
Ruby installer script with 13 lines of makefile code: 
https://github.com/D-Programming-Language/installer/pull/10/files. It 
would be difficult to construct an argument against that work.

Ruby and Python have all my respect as an outsider of their respective 
communities: they have users who enjoy them and get work done using 
them. That's always a good thing in my book.

That said, I wouldn't feel easy appealing to Ruby or Python for D 
internal work for reasons that I consider entirely practical and 
non-political:

* One more language for the maintainers to know and use.

* One more dependency. Although scripting languages are ubiquitous 
enough, I can tell from direct experience that versioning and dependent 
packages can be quite a hassle.

* Escaping into a high-level language seems as much "cheating" as 
escaping into a low-level language. If C or C++ would be needed instead 
of D for a task, it is worthwhile exploring how to make D a better 
replacement for them . This has been historically a good and important 
goal to pursue. Similarly I'd rather explore what it takes to expand D 
into high-level territory instead of escaping into a high-level language.


Andrei
Feb 08 2015
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-02-08 17:57, Andrei Alexandrescu wrote:


 Ehm. This part sounds unnecessarily a bit political - NIH syndrome,
 closemindedness of the D folks toward using anything else but D and
 make, and such.


I'm just sharing my experience since I've tried this before. I would not 
recommend using any other language than D unless it has the community's 
blessing. I'm just trying to save him the trouble.


 I do remember one action I took "against" Ruby - replacing a 109 line
 Ruby installer script with 13 lines of makefile code:
 https://github.com/D-Programming-Language/installer/pull/10/files. It
 would be difficult to construct an argument against that work.


I specially remember you saying something along the lines that it took 
me two years to consider D instead of Ruby for Orbit.


 Ruby and Python have all my respect as an outsider of their respective
 communities: they have users who enjoy them and get work done using
 them. That's always a good thing in my book.

 That said, I wouldn't feel easy appealing to Ruby or Python for D
 internal work for reasons that I consider entirely practical and
 non-political:

 * One more language for the maintainers to know and use.


Same thing with make or shell scripts.


 * One more dependency. Although scripting languages are ubiquitous
 enough, I can tell from direct experience that versioning and dependent
 packages can be quite a hassle.


Only for building the tool. The scripting language would be built-in to 
the executable.


 * Escaping into a high-level language seems as much "cheating" as
 escaping into a low-level language. If C or C++ would be needed instead
 of D for a task, it is worthwhile exploring how to make D a better
 replacement for them . This has been historically a good and important
 goal to pursue. Similarly I'd rather explore what it takes to expand D
 into high-level territory instead of escaping into a high-level language.


It's just that the syntax Ruby has, in my opinion, is better suited for 
a declarative DSL, i.e. call methods without parentheses, 
blocks/trailing delegates, top level execution of code.

-- 
/Jacob Carlborg
Feb 08 2015
prev sibling parent reply Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 2015-02-08 at 08:57 -0800, Andrei Alexandrescu via Digitalmars-d
wrote:
[=E2=80=A6]


 * One more language for the maintainers to know and use.


On the other hand by replacing Make you lose two languages, so total one
less language to know.


 * One more dependency. Although scripting languages are ubiquitous=20
 enough, I can tell from direct experience that versioning and dependent=


=20

 packages can be quite a hassle.


This applies to the entire D infrastructure (and also the C, C++, Make,
Bash,..), versioning in all systems is currently a serious problem,
possibly insoluble, so this would not be a new thing at all.


 * Escaping into a high-level language seems as much "cheating" as=20
 escaping into a low-level language. If C or C++ would be needed instead=


=20

 of D for a task, it is worthwhile exploring how to make D a better=20
 replacement for them . This has been historically a good and important=


=20

 goal to pursue. Similarly I'd rather explore what it takes to expand D=


=20

 into high-level territory instead of escaping into a high-level language.


I definitely agree this is a good thing, but I have yet to see a good
build system with serious traction that is purely statically typed and
compiled. Maybe D could be different. Perhaps another GSoC 2015 project
in here?


--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Feb 09 2015
next sibling parent "Atila Neves" <atila.neves gmail.com> writes:
 I definitely agree this is a good thing, but I have yet to see 
 a good
 build system with serious traction that is purely statically 
 typed and
 compiled. Maybe D could be different. Perhaps another GSoC 2015 
 project
 in here?


I haven't tried Shake, but since it's in Haskell...

Atila
Feb 09 2015
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/9/15 12:43 AM, Russel Winder via Digitalmars-d wrote:

 On Sun, 2015-02-08 at 08:57 -0800, Andrei Alexandrescu via Digitalmars-d
 wrote:
 […]


 * One more language for the maintainers to know and use.


 On the other hand by replacing Make you lose two languages, so total one
 less language to know.


That's not elastic. By eliminating make/sh we don't automatically 
"forget" them making place for others :o).


 * One more dependency. Although scripting languages are ubiquitous
 enough, I can tell from direct experience that versioning and dependent
 packages can be quite a hassle.


 This applies to the entire D infrastructure (and also the C, C++, Make,
 Bash,..), versioning in all systems is currently a serious problem,
 possibly insoluble, so this would not be a new thing at all.


Fewer is better. "You have a pet in the house already, so you know what 
it takes to keep one. Take mine too..."?


 * Escaping into a high-level language seems as much "cheating" as
 escaping into a low-level language. If C or C++ would be needed instead
 of D for a task, it is worthwhile exploring how to make D a better
 replacement for them . This has been historically a good and important
 goal to pursue. Similarly I'd rather explore what it takes to expand D
 into high-level territory instead of escaping into a high-level language.


 I definitely agree this is a good thing, but I have yet to see a good
 build system with serious traction that is purely statically typed and
 compiled. Maybe D could be different. Perhaps another GSoC 2015 project
 in here?


That would be interesting.


Andrei
Feb 09 2015
next sibling parent "CraigDillabaugh" <craig.dillabaugh gmail.com> writes:
On Monday, 9 February 2015 at 17:43:58 UTC, Andrei Alexandrescu 
wrote:

 On 2/9/15 12:43 AM, Russel Winder via Digitalmars-d wrote:


clip

 I definitely agree this is a good thing, but I have yet to see 
 a good
 build system with serious traction that is purely statically 
 typed and
 compiled. Maybe D could be different. Perhaps another GSoC 
 2015 project
 in here?


 That would be interesting.


 Andrei


There is still time!
Feb 09 2015
prev sibling parent "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
They are MakefileForD

https://github.com/bioinfornatics/MakefileForD

It is easy to use. You have only to set First lines:

export PROJECT_NAME =
export AUTHOR	   =
export DESCRIPTION  =
export VERSION	  =
export LICENSE	  =
ROOT_SOURCE_DIR	 =
EXE_NAME =

and that will find automatically file to build and install it 
with respect of the filesystem hierarchy standard (FHS).

Packager will love it is customizable as a packager want to be 
(not dub)
with standard var:
- INCLUDE_DIR
- BIN_DIR
- LIB_DIR
- DATA_DIR
- DESTDIR

Yes is not a D program but at least is well documented, used 
everywhere and does the job easily, any linux has make...
Feb 09 2015
prev sibling parent reply "Tobias Pankrath" <tobias pankrath.net> writes:
On Monday, 2 February 2015 at 16:42:03 UTC, Dicebot wrote:

 On Monday, 2 February 2015 at 16:26:45 UTC, Atila Neves wrote:

 This thread has just made me decide to write a D build system, 
 in D and configured in D that builds on dub and its packages 
 intead of reinventing the wheel. I might ask the community for 
 inputs on requirements. Maybe this will be my DConf 2015 
 submission.


 I have recently been experimenting with meta-dlang repository 
 (to aggregate exisiting ones) and using D scripts instead of 
 makefiles for uniform cross-platform build experience.

 That quickly has made me wanting to get something like 
 `std.make` into Phobos - small utility module that would allow 
 declaratively defining target tree + shell scripts similar to 
 makefiles, as well as set of small nice shell wrappers (like 
 one that effectively enables `set -e` for called commands). 
 Having such module in standard library would make it much 
 easier to replace makefiles with a quick D scripts and 
 potentially build a robust build system on top.


https://github.com/abscissa/scriptlike
Feb 02 2015
next sibling parent "Dicebot" <public dicebot.lv> writes:
On Monday, 2 February 2015 at 16:56:52 UTC, Tobias Pankrath wrote:

 https://github.com/abscissa/scriptlike


Yes, there is plenty of good stuff there and it can be used as 
inspiration. Though AFAIK it misses that bit about defining 
action/target tree and automatically exposing it to CLI (which is 
not something std.getopt helps with)

Getting it into Phobos is important though as that would mean 
that any project that uses D for scripting would only need most 
basic working D installation for bootstrapping - no controversy 
about dependency management or installation of extra tools.

I have started poking too many things at once and don't want to 
commit myself to implementing such a module but if something is 
willing to use Nick Sabalausky as base and create Phobos proposal 
from it, I will gladly help in both implementation and pushing it 
through formal review queue.
Feb 02 2015
prev sibling parent reply "Jonathan Marler" <johnnymarler gmail.com> writes:
On Monday, 2 February 2015 at 16:56:52 UTC, Tobias Pankrath wrote:

 On Monday, 2 February 2015 at 16:42:03 UTC, Dicebot wrote:

 On Monday, 2 February 2015 at 16:26:45 UTC, Atila Neves wrote:

 This thread has just made me decide to write a D build 
 system, in D and configured in D that builds on dub and its 
 packages intead of reinventing the wheel. I might ask the 
 community for inputs on requirements. Maybe this will be my 
 DConf 2015 submission.


 I have recently been experimenting with meta-dlang repository 
 (to aggregate exisiting ones) and using D scripts instead of 
 makefiles for uniform cross-platform build experience.

 That quickly has made me wanting to get something like 
 `std.make` into Phobos - small utility module that would allow 
 declaratively defining target tree + shell scripts similar to 
 makefiles, as well as set of small nice shell wrappers (like 
 one that effectively enables `set -e` for called commands). 
 Having such module in standard library would make it much 
 easier to replace makefiles with a quick D scripts and 
 potentially build a robust build system on top.


 https://github.com/abscissa/scriptlike


Are you the owner of this code?  If you are could you put some 
examples of it in the readme/api docs.  Examples that demonstrate 
the use cases that this library makes very easy.
Feb 02 2015
next sibling parent "Tobias Pankrath" <tobias pankrath.net> writes:
On Monday, 2 February 2015 at 18:29:40 UTC, Jonathan Marler wrote:

 scripts and potentially build a robust build system on top.


 https://github.com/abscissa/scriptlike


 Are you the owner of this code?  If you are could you put some 
 examples of it in the readme/api docs.  Examples that 
 demonstrate the use cases that this library makes very easy.


Nope.
Feb 02 2015
prev sibling parent "Dicebot" <public dicebot.lv> writes:
On Monday, 2 February 2015 at 18:29:40 UTC, Jonathan Marler wrote:

 https://github.com/abscissa/scriptlike


 Are you the owner of this code?  If you are could you put some 
 examples of it in the readme/api docs.  Examples that 
 demonstrate the use cases that this library makes very easy.


abscissa is Nick Sabalausky
Feb 02 2015
prev sibling parent reply "Dragos Carp" <dragoscarp gmail.com> writes:
On Monday, 2 February 2015 at 16:26:45 UTC, Atila Neves wrote:

 This thread has just made me decide to write a D build system, 
 in D and configured in D that builds on dub and its packages 
 intead of reinventing the wheel. I might ask the community for 
 inputs on requirements. Maybe this will be my DConf 2015 
 submission.


Why not building on cmake? You have experience with it. 
trentforkert made very good progress with D support in cmake and 
has good chances to merge it upstream [1].

If we look at the Vision/2015H1 goals, we see a couple of points, 
where cmake already has the necessary support:

- C++ integration (hybrid projects)
- alternative compilers
- embedded systems

This features would take a lot of effort/time to be implemented 
in a pure D solution, nevertheless any of these points is a niche 
in the D community.

Apart from the ugly script language, cmake would be a pretty good 
fit.


[1] - https://github.com/trentforkert/cmake
Feb 02 2015
next sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
I wasn't forgetting C++ integration, if I do this that'd be a 
main part of the build system.

Ugly doesn't do the CMake scripting language justice. CMake is 
like democracy; terrible, but better than everything else. I 
think I can do better but I need to figure out a bunch of details.

Atila


On Monday, 2 February 2015 at 20:05:38 UTC, Dragos Carp wrote:

 On Monday, 2 February 2015 at 16:26:45 UTC, Atila Neves wrote:

 This thread has just made me decide to write a D build system, 
 in D and configured in D that builds on dub and its packages 
 intead of reinventing the wheel. I might ask the community for 
 inputs on requirements. Maybe this will be my DConf 2015 
 submission.


 Why not building on cmake? You have experience with it. 
 trentforkert made very good progress with D support in cmake 
 and has good chances to merge it upstream [1].

 If we look at the Vision/2015H1 goals, we see a couple of 
 points, where cmake already has the necessary support:

 - C++ integration (hybrid projects)
 - alternative compilers
 - embedded systems

 This features would take a lot of effort/time to be implemented 
 in a pure D solution, nevertheless any of these points is a 
 niche in the D community.

 Apart from the ugly script language, cmake would be a pretty 
 good fit.


 [1] - https://github.com/trentforkert/cmake
Feb 03 2015
next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 3 February 2015 at 08:39:56 UTC, Atila Neves wrote:

 I wasn't forgetting C++ integration, if I do this that'd be a 
 main part of the build system.

 Ugly doesn't do the CMake scripting language justice. CMake is 
 like democracy; terrible, but better than everything else. I 
 think I can do better but I need to figure out a bunch of 
 details.


You are obviously biased as you live in Switzerland. This is 
literally the only democracy that is working :)
Feb 03 2015
parent "Joseph Rushton Wakeling" <joseph.wakeling webdrake.net> writes:
On Tuesday, 3 February 2015 at 09:28:36 UTC, deadalnix wrote:

 You are obviously biased as you live in Switzerland. This is 
 literally the only democracy that is working :)


People living in Switzerland might dispute this ;-)
Feb 03 2015
prev sibling parent reply "Tofu Ninja" <emmons0 purdue.edu> writes:
On Tuesday, 3 February 2015 at 08:39:56 UTC, Atila Neves wrote:

 I wasn't forgetting C++ integration, if I do this that'd be a 
 main part of the build system.

 Ugly doesn't do the CMake scripting language justice. CMake is 
 like democracy; terrible, but better than everything else. I 
 think I can do better but I need to figure out a bunch of 
 details.

 Atila


Why not invest your time into improving dub and adding these 
features, we don't need another tool. Improving dub would have a 
much bigger impact than making another tool that is almost 
certainly never going to get used because very few tools ever get 
to that level.

Honestly it seems like a huge waist of time and very counter 
productive.
Feb 03 2015
next sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
It's easier to get started without getting tied down by an 
existing codebase, for one. But you've got me thinking: if I do 
anything I should do it as a library first and foremost to make 
its possible future inclusion into dub a lot easier. I hadn't 
considered that.

I was also thinking of not adding complexity to dub, but I guess 
most other language-specific package managers do double duty as 
build tools as well. How well they do that moonlighting job is 
another matter.

In any case, I don't want to replace dub; I want to use it for 
package dependencies for those who want to build their software 
that way. I also want to enable local filepaths and 
github/bitbucket/whatever direct links, kind of like what "go 
get" does, for use-cases like Vladimir's. Oh, and C/C++ 
integration. That's the idea I had yesterday anyway.

Atila



 Why not invest your time into improving dub and adding these 
 features, we don't need another tool. Improving dub would have 
 a much bigger impact than making another tool that is almost 
 certainly never going to get used because very few tools ever 
 get to that level.

 Honestly it seems like a huge waist of time and very counter 
 productive.
Feb 03 2015
parent "Tofu Ninja" <emmons0 purdue.edu> writes:
On Tuesday, 3 February 2015 at 15:17:21 UTC, Atila Neves wrote:

 It's easier to get started without getting tied down by an 
 existing codebase, for one. But you've got me thinking: if I do 
 anything I should do it as a library first and foremost to make 
 its possible future inclusion into dub a lot easier. I hadn't 
 considered that.


This seems like a good tactic to obviate the need to deal with a 
foreign code base, but I still think that the plan should be to 
eventually try to integrate it into dub and that its design 
should reflect that.


 I was also thinking of not adding complexity to dub, but I 
 guess most other language-specific package managers do double 
 duty as build tools as well. How well they do that moonlighting 
 job is another matter.


The problem is that while packager management and build tools are 
different problems, they are also very highly connected. A lot of 
the information needs to be shared between them both and the 
existence of one basically requires the existence of the 
other(whats the point of a package manager if you cant build?). 
If combining them eases the interaction between them, then I am 
all for it.


 In any case, I don't want to replace dub; I want to use it for 
 package dependencies for those who want to build their software 
 that way. I also want to enable local filepaths and 
 github/bitbucket/whatever direct links, kind of like what "go 
 get" does, for use-cases like Vladimir's. Oh, and C/C++ 
 integration. That's the idea I had yesterday anyway.


I think C and C++ are the only ones that matter and the only ones 
that people would realistically care about for a D package 
manager/build tools. If there was a clean way to integrate C++ 
into a dub project with as little fuss as possible, then I think 
it would be a huge win. And it fits nicely with the C++ 
compatibility focus that has been a big thing recently.
Feb 03 2015
prev sibling parent reply Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 2015-02-03 at 13:19 +0000, Tofu Ninja via Digitalmars-d wrote:
[…]

 Why not invest your time into improving dub and adding these 
 features, we don't need another tool. Improving dub would have a 
 much bigger impact than making another tool that is almost 
 certainly never going to get used because very few tools ever get 
 to that level.


Because they feel that Dub is fundamentally flawed as the future
solution to D (C++/C) build?

 
 Honestly it seems like a huge waist of time and very counter 
 productive.


I disagree. This sort of argument was made before Dub and yet Dub
happened. 

If people want to improve on Dub and replace it by doing exactly that,
that seems like the best way forward.

I have just realized why I think Dub will not be the future of D build…

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Feb 03 2015
parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 03.02.2015 um 19:07 schrieb Russel Winder via Digitalmars-d:

 On Tue, 2015-02-03 at 13:19 +0000, Tofu Ninja via Digitalmars-d wrote:
 […]

 Why not invest your time into improving dub and adding these
 features, we don't need another tool. Improving dub would have a
 much bigger impact than making another tool that is almost
 certainly never going to get used because very few tools ever get
 to that level.


 Because they feel that Dub is fundamentally flawed as the future
 solution to D (C++/C) build?

 Honestly it seems like a huge waist of time and very counter
 productive.


 I disagree. This sort of argument was made before Dub and yet Dub
 happened.

 If people want to improve on Dub and replace it by doing exactly that,
 that seems like the best way forward.

 I have just realized why I think Dub will not be the future of D build…


Thanks for that kick in the teeth. I'm now just left wondering what it 
is that is "fundamentally flawed". Everything mentioned so far is either 
bugs or missing functionality, or rather just mostly missing convenience.

Things may not (yet) be ideal, but fundamentally DUB is nothing more 
than a tool built on a defined abstract package description format (vs. 
a procedural build description). If you (or anyone else for that matter) 
want to say that using a descriptive format is fundamentally flawed then 
I'd be grateful for some additional substantiation. Otherwise there is 
nothing "fundamental" in the system that I know of.
Feb 05 2015
parent reply Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Thu, 2015-02-05 at 18:01 +0100, S=C3=B6nke Ludwig via Digitalmars-d wrot=
e:

=20


[=E2=80=A6]

  I'm now just left wondering what it is that is "fundamentally=20
 flawed". Everything mentioned so far is either
 bugs or missing functionality, or rather just mostly missing=20
 convenience.


The core problem is the use of a JSON file to express the definition=20
of the project. This sort of specification is fine for dependency but=20
not sufficient for build.

Make is a fine external DSL for dependencies, but everything else=20
relies on the action language, generally Bash. GNU Make adds macros=20
and a whole slew of things to try and make things better, all of them=20
adding more imperative aspects.

Autotools is a wonderful M4 edifice, but not a solution to the=20
problems of project management Make despite many years of people=20
actually succeeding in doing it.

CMake has a DSL for describing build, not so good on the project=20
description side, but in the end the language is ugly, underpowered=20
and reliant on Make.

SCons is not perfect, but it beats Make and CMake by creating an=20
internal DSL on Python. Waf takes aspects of this further and makes=20
Autotools look a bit ill. There is a natural balance=20

Go delegates to a DVCS all dependency issues, everything build-wise is=20
totally by convention. It works but the hassle people are finding with=20
repeatable builds leads to conventions as per gopkg.in.

Ant, hummm=E2=80=A6 started out as a platform independent version on Make,=
=20
used XML for build specification and claimed to be declarative.=20
Looking at Ant scripts these days, the hoops people go through to=20
construct imperative programs in XML are amazing. The only alternative=20
is to write a new task, which lots of people have to do.

Maven tried to go the project specification, and partly succeeded.=20
Artefact repository approach a huge win. Specification in XML, yuk.=20
Attempt at totally convention based build only partly worked. Lots of=20
people have to write plugins to get the process they need for their=20
goals.

Gradle fixed the transitive dependency tracking problems of Maven and=20
has Groovy specifications. Yes there is a lot of purely declarative=20
convention-based stuff, quite right too, but the ability to trivially=20
add the extra bit of imperative in the build specification, stops the=20
mess of Maven, Ant, CMake, Make.

SBT tries to use Scala as a project and build specification language.=20
For simple projects, a few declarative lines are all that is needed.=20
However for anything complicated you end up going outside the=20
convention-only, but like Gradle and unlike Maven you can do this.=20
SBTs major problem is Scala compile time for non-trivial project=20
specification. Hence the need for a build daemon, a route Gradle has=20
had to go as well due to infrastructure start-up time.

I could turn this into a better essay, but the above gives the main=20
threads of the argument: project specification can be declarative,=20
build specification is best as declarative as possible, convention-
based, but not totally declarative.

(I now I should mention tup here but I still haven't used it enough to=20
know anything real about it.)


 Things may not (yet) be ideal, but fundamentally DUB is nothing more=20
 than a tool built on a defined abstract package description format=20
 (vs. a procedural build description). If you (or anyone else for=20
 that matter)
 want to say that using a descriptive format is fundamentally flawed=20
 then
 I'd be grateful for some additional substantiation. Otherwise there=20
 is nothing "fundamental" in the system that I know of.


The fundamental problem here is that project specification and build=20
specification is not either declarative or imperative.  From 1977=20
onwards we have a trail of evidence that shows it is part declarative=20
(as much as possible), but part imperative (as little as possible, but=20
necessary for non-trivial projects).

Sticking with JSON as the only specification notation for Dub will=20
inevitably lead people to have to hack or find another tool. The=20
solution would be to either:

=E2=80=93 Treat the JSON specification for dependency management only and u=
se=20
something else for build description so as not to be constrained by=20
the only convention.=20

=E2=80=93 switch to an internal DSL so as to have mostly declarative but=
=20
imperative as needed description of project, dependency and build=20
process.


--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Feb 06 2015
next sibling parent reply "Dicebot" <public dicebot.lv> writes:
I believe only truly practical approach is to design a D build 
tool as a library (and, ideally, make it a Phobos module). But at 
the same time it is important to keep it generic and not tied to 
D or building application in general.

There are two important bits here (important for me):

1)

It must be designed in terms of "target - script - dependency" 
abstractions, similar to make or tup. I will never use anything 
that pretends to be a build tool but keeps imagining build 
process as "compile and link bunch of files and libraries". Good 
litmus test is this makefile target chain we have in one of 
projects (very simplified):

bin/app: src/app/main.d protocol.o


src/app/main.d: $(D_SOURCES)

protocol.d: protocol.h


protocol.h protocol.c: protocol.proto


Anything that does not allow me to express such dependency chain 
in native straightforward manner without resorting to external 
scripts is simply not good enough. dub fails this, CMake fails 
this.

2)

D is only cross-platfrom scripting language we can rely on. This 
is probably biggest problem of make (apart from bunch of legacy 
syntax crap) - any real build system needs relatively complicated 
build rules for target transformation. However we can reasonably 
expect working D compiler from anyone wanting to compile D 
project - which is both perfectly cross-platform and does not 
request installation of any additional binaries.
Feb 06 2015
next sibling parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Dicebot"  wrote in message news:listanyegqyanevsnvlv forum.dlang.org... 


 D is only cross-platfrom scripting language we can rely on. This 
 is probably biggest problem of make (apart from bunch of legacy 
 syntax crap) - any real build system needs relatively complicated 
 build rules for target transformation. However we can reasonably 
 expect working D compiler from anyone wanting to compile D 
 project - which is both perfectly cross-platform and does not 
 request installation of any additional binaries.


Sometimes you make a lot of sense.
Feb 06 2015
next sibling parent "Paolo Invernizzi" <paolo.invernizzi no.address> writes:
On Friday, 6 February 2015 at 12:30:45 UTC, Daniel Murphy wrote:

 "Dicebot"  wrote in message 
 news:listanyegqyanevsnvlv forum.dlang.org...


 D is only cross-platfrom scripting language we can rely on. 
 This is probably biggest problem of make (apart from bunch of 
 legacy syntax crap) - any real build system needs relatively 
 complicated build rules for target transformation. However we 
 can reasonably expect working D compiler from anyone wanting 
 to compile D project - which is both perfectly cross-platform 
 and does not request installation of any additional binaries.


 Sometimes you make a lot of sense.


LOL
Feb 06 2015
prev sibling parent "Dicebot" <public dicebot.lv> writes:
On Friday, 6 February 2015 at 12:30:45 UTC, Daniel Murphy wrote:

 "Dicebot"  wrote in message 
 news:listanyegqyanevsnvlv forum.dlang.org...


 D is only cross-platfrom scripting language we can rely on. 
 This is probably biggest problem of make (apart from bunch of 
 legacy syntax crap) - any real build system needs relatively 
 complicated build rules for target transformation. However we 
 can reasonably expect working D compiler from anyone wanting 
 to compile D project - which is both perfectly cross-platform 
 and does not request installation of any additional binaries.


 Sometimes you make a lot of sense.


That is purely accidental - hard to disappoint everyone all the 
time!
Feb 06 2015
prev sibling next sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
 There are two important bits here (important for me):


The approach I'm thinking of agrees 100% with your two important 
bits.


 Anything that does not allow me to express such dependency 
 chain in native straightforward manner without resorting to 
 external scripts is simply not good enough. dub fails this, 
 CMake fails this.


CMake doesn't fail at this, it's just not easy. It's what 
add_custom_command is for.


 D is only cross-platfrom scripting language we can rely on.


Which is why that's the language I want to express the build in. 
Methinks I have some code to write and some emailing to do with 
Soenke.

Atila
Feb 06 2015
next sibling parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Fri, 2015-02-06 at 13:35 +0000, Atila Neves via Digitalmars-d wrote:

=20




[=E2=80=A6]

 Which is why that's the language I want to express the build in.=20
 Methinks I have some code to write and some emailing to do with=20
 Soenke.


In this situation it would be good having many people working the same=20
infrastructure. The time for lots of individual projects on this=20
aspect of D is long gone.

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Feb 06 2015
prev sibling parent "Dicebot" <public dicebot.lv> writes:
On Friday, 6 February 2015 at 13:35:43 UTC, Atila Neves wrote:

 There are two important bits here (important for me):


 The approach I'm thinking of agrees 100% with your two 
 important bits.


 Anything that does not allow me to express such dependency 
 chain in native straightforward manner without resorting to 
 external scripts is simply not good enough. dub fails this, 
 CMake fails this.


 CMake doesn't fail at this, it's just not easy. It's what 
 add_custom_command is for.


"not easy" == "fails" in my opinion :)


 D is only cross-platfrom scripting language we can rely on.


 Which is why that's the language I want to express the build 
 in. Methinks I have some code to write and some emailing to do 
 with Soenke.


Looking forward to hearing the outcome of this ^_^
Feb 06 2015
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-02-06 12:37, Dicebot wrote:


 D is only cross-platfrom scripting language we can rely on. This is
 probably biggest problem of make (apart from bunch of legacy syntax
 crap) - any real build system needs relatively complicated build rules
 for target transformation. However we can reasonably expect working D
 compiler from anyone wanting to compile D project - which is both
 perfectly cross-platform and does not request installation of any
 additional binaries.


While this is true, in my opinion, the D syntax is not very good for a 
declarative DSL.

Most scripting language can be bundle with the build tool, that is 
linked in the executable. Ruby is my favorite because it allows to 
create good declarative DSL's, but I know most developers here hate it.

-- 
/Jacob Carlborg
Feb 08 2015
parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Sunday, 8 February 2015 at 17:59:56 UTC, Jacob Carlborg wrote:

 On 2015-02-06 12:37, Dicebot wrote:


 D is only cross-platfrom scripting language we can rely on. 
 This is
 probably biggest problem of make (apart from bunch of legacy 
 syntax
 crap) - any real build system needs relatively complicated 
 build rules
 for target transformation. However we can reasonably expect 
 working D
 compiler from anyone wanting to compile D project - which is 
 both
 perfectly cross-platform and does not request installation of 
 any
 additional binaries.


 While this is true, in my opinion, the D syntax is not very 
 good for a declarative DSL.

 Most scripting language can be bundle with the build tool, that 
 is linked in the executable. Ruby is my favorite because it 
 allows to create good declarative DSL's, but I know most 
 developers here hate it.


I don't hate Ruby, I just would like that the community would
look at languages like Dylan and provide proper JIT and AOT
compilers.

Instead there is RubyMotion, which gets some heat of the
community AFAIK and focus only on iOS.

Since my TCL/Python days I am no longer into languages where JIT
or AOT isn't part of the reference platform.

--
Paulo
Feb 08 2015
prev sibling next sibling parent "Atila Neves" <atila.neves gmail.com> writes:
+1 to all of this. Except for the part about CMake relying on 
Make. It doesn't, Make is just one of the backends. I haven't 
used CMake with anything other than Ninja in a very long time.

Atila

On Friday, 6 February 2015 at 11:04:59 UTC, Russel Winder wrote:

 On Thu, 2015-02-05 at 18:01 +0100, Sönke Ludwig via 
 Digitalmars-d wrote:

 


 […]

  I'm now just left wondering what it is that is "fundamentally 
 flawed". Everything mentioned so far is either
 bugs or missing functionality, or rather just mostly missing 
 convenience.


 The core problem is the use of a JSON file to express the 
 definition
 of the project. This sort of specification is fine for 
 dependency but
 not sufficient for build.

 Make is a fine external DSL for dependencies, but everything 
 else
 relies on the action language, generally Bash. GNU Make adds 
 macros
 and a whole slew of things to try and make things better, all 
 of them
 adding more imperative aspects.

 Autotools is a wonderful M4 edifice, but not a solution to the
 problems of project management Make despite many years of people
 actually succeeding in doing it.

 CMake has a DSL for describing build, not so good on the project
 description side, but in the end the language is ugly, 
 underpowered
 and reliant on Make.

 SCons is not perfect, but it beats Make and CMake by creating an
 internal DSL on Python. Waf takes aspects of this further and 
 makes
 Autotools look a bit ill. There is a natural balance

 Go delegates to a DVCS all dependency issues, everything 
 build-wise is
 totally by convention. It works but the hassle people are 
 finding with
 repeatable builds leads to conventions as per gopkg.in.

 Ant, hummm… started out as a platform independent version on 
 Make,
 used XML for build specification and claimed to be declarative.
 Looking at Ant scripts these days, the hoops people go through 
 to
 construct imperative programs in XML are amazing. The only 
 alternative
 is to write a new task, which lots of people have to do.

 Maven tried to go the project specification, and partly 
 succeeded.
 Artefact repository approach a huge win. Specification in XML, 
 yuk.
 Attempt at totally convention based build only partly worked. 
 Lots of
 people have to write plugins to get the process they need for 
 their
 goals.

 Gradle fixed the transitive dependency tracking problems of 
 Maven and
 has Groovy specifications. Yes there is a lot of purely 
 declarative
 convention-based stuff, quite right too, but the ability to 
 trivially
 add the extra bit of imperative in the build specification, 
 stops the
 mess of Maven, Ant, CMake, Make.

 SBT tries to use Scala as a project and build specification 
 language.
 For simple projects, a few declarative lines are all that is 
 needed.
 However for anything complicated you end up going outside the
 convention-only, but like Gradle and unlike Maven you can do 
 this.
 SBTs major problem is Scala compile time for non-trivial project
 specification. Hence the need for a build daemon, a route 
 Gradle has
 had to go as well due to infrastructure start-up time.

 I could turn this into a better essay, but the above gives the 
 main
 threads of the argument: project specification can be 
 declarative,
 build specification is best as declarative as possible, 
 convention-
 based, but not totally declarative.

 (I now I should mention tup here but I still haven't used it 
 enough to
 know anything real about it.)


 Things may not (yet) be ideal, but fundamentally DUB is 
 nothing more than a tool built on a defined abstract package 
 description format (vs. a procedural build description). If 
 you (or anyone else for that matter)
 want to say that using a descriptive format is fundamentally 
 flawed then
 I'd be grateful for some additional substantiation. Otherwise 
 there is nothing "fundamental" in the system that I know of.


 The fundamental problem here is that project specification and 
 build
 specification is not either declarative or imperative.  From 
 1977
 onwards we have a trail of evidence that shows it is part 
 declarative
 (as much as possible), but part imperative (as little as 
 possible, but
 necessary for non-trivial projects).

 Sticking with JSON as the only specification notation for Dub 
 will
 inevitably lead people to have to hack or find another tool. The
 solution would be to either:

 – Treat the JSON specification for dependency management only 
 and use
 something else for build description so as not to be 
 constrained by
 the only convention.

 – switch to an internal DSL so as to have mostly declarative but
 imperative as needed description of project, dependency and 
 build
 process.
Feb 06 2015
prev sibling parent =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
It looks like the core issue is that we simply have different goals in 
mind. When we were deciding on how the build description works back 
then, we basically had these:

  - The build description can be reasoned about in a generic way (not 
generally possible with an imperative scripting language)
  - It avoids security issues when doing so (DoS, attack surface for 
more sophisticated attacks), so that it can be done on a server with 
decent confidence and without expensive safeguards
  - It avoids dependencies on the host environment, so that reasoning 
about the build description can be done independently of the target 
platform/environment
  - Should work seamlessly for 99% of the projects, but allows to invoke 
external tools to handle the rest
  - Avoids any form of redundant information as far as possible (such as 
compiler or OS specific compiler flags)
  - Can be mapped to the typical IDE project file types, as well as to 
other kinds of build descriptions

The first goals are only achievable with either a declarative approach, 
or with a very limited imperative approach (which in the end is no more 
powerful than the declarative one). I also think that we can easily 
achieve the 99%/1% goal (if that has been reached already is difficult 
to tell, but with C-class language support it will most definitely be), 
and if we do, it's hard to justify why this approach with limited 
expressibility should be regarded as a failure.

I'm not sure why some people insist that it must be possible to achieve 
everything with one tool. Of course that is a noble goal and definitely 
can have its beauty, but, assuming that the 99%/1% rule holds, it may 
not be one that has much importance in practice (other than producing 
controversy). On the other hand, being able to reason about the build 
description without actually executing a build script on each of the 
interesting platforms/environments can be a big win and we'd otherwise 
lose some possibly important automation opportunities (mostly when 
talking about a public web service scale).

For me, the current solution seems to be a good trade off, ignoring the 
missing support for other languages for a moment. It has successfully 
laid the basis for a constantly growing ecosystem of pure D packages and 
now it's time to carefully support the more advanced use cases.

In this process, we can of course always go through the initial goals 
and revise them where appropriate, or evaluate any other possible 
approaches that can fulfill those goals. But we have to be very careful 
to not make the mistake and disrupt the existing ecosystem. Backwards 
compatibility is a must, and ideally any possible new approach should 
fit well with the existing system (it doesn't have to fit well with the 
JSON format, though, it just has to fit somehow).

One possible alternative to a full procedural build description could be 
the introduction of a plugin system, where each plugin is written in D 
and can be invoked from within the package description. I personally 
would like to let the selection of an approach be guided by actual use 
cases instead of just focusing on the maximum expressibility.
Feb 09 2015
prev sibling next sibling parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, 2015-02-02 at 20:05 +0000, Dragos Carp via Digitalmars-d wrote:
[…]

 Why not building on cmake? You have experience with it. 


I would argue because the CMake build description language really,
really suck.

Yes, you can describe builds using it, but only once you have entered a
state of Stockholm Syndrome.


 trentforkert made very good progress with D support in cmake and 
 has good chances to merge it upstream [1].


OK, positive progress is good progress. Making a bad system better is
better than leaving it bad.


 If we look at the Vision/2015H1 goals, we see a couple of points, 
 where cmake already has the necessary support:
 
 - C++ integration (hybrid projects)
 - alternative compilers
 - embedded systems
 
 This features would take a lot of effort/time to be implemented 
 in a pure D solution, nevertheless any of these points is a niche 
 in the D community.
 
 Apart from the ugly script language, cmake would be a pretty good 
 fit.


As would SCons. 

(You just knew I was going to say that didn't you :-)


 
 [1] - https://github.com/trentforkert/cmake


-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Feb 03 2015
prev sibling parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 02.02.2015 um 21:05 schrieb Dragos Carp:

 On Monday, 2 February 2015 at 16:26:45 UTC, Atila Neves wrote:

 This thread has just made me decide to write a D build system, in D
 and configured in D that builds on dub and its packages intead of
 reinventing the wheel. I might ask the community for inputs on
 requirements. Maybe this will be my DConf 2015 submission.


 Why not building on cmake? You have experience with it. trentforkert
 made very good progress with D support in cmake and has good chances to
 merge it upstream [1].

 If we look at the Vision/2015H1 goals, we see a couple of points, where
 cmake already has the necessary support:

 - C++ integration (hybrid projects)
 - alternative compilers
 - embedded systems

 This features would take a lot of effort/time to be implemented in a
 pure D solution, nevertheless any of these points is a niche in the D
 community.

 Apart from the ugly script language, cmake would be a pretty good fit.


 [1] - https://github.com/trentforkert/cmake


BTW, CMake output support (using "dub generate cmake") has recently been 
added to GIT master.
Feb 05 2015
parent "Dragos Carp" <dragoscarp gmail.com> writes:
 BTW, CMake output support (using "dub generate cmake") has 
 recently been added to GIT master.


Very nice!
Feb 05 2015
prev sibling next sibling parent Joseph Rushton Wakeling via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 02/02/15 15:40, Dicebot via Digitalmars-d wrote:

 Quick summary of my opinion can look like this:

 1) dub is a horrible build tool
 2) git submodules is a horrible way for dependency management

 First thing is something we can improve on in future and, actually, dub
supports
 using rdmd as a build tool already.

 Second thing is an inherent issue that won't change without some fundamental
 breaking changes in how hit submodules are implemented.

 Because of that I am trying to push dub despite the fact I don't like lot of
 things about it personally.


Very well put.  If there are issues with dub, we can and should address them -- 
not turn to a tool that isn't actually intended to solve the problem we are 
trying to address.


 4) Inherently coupled with specific DVCS

 git and GitHub are most common (and officially endorsed) ways of version
control
 in D world but those are not the only one. dub registry allows to use GitHub,
 GitBucket and source tarball packages in the same project - in simple uniform
 matter. It is a 100% in-house tool that does no depend on decisions of external
 project maintainers that provides a common API for everything else to
interoperate


Yup.  It would be very presumptuous to assume that just because git works well 
for the core D project, that everyone out there should use it for their
projects.

By the way, talking of other VCS, I recently discovered a neat little trick for 
visualizing a complex git history in a "clean" way.  You need to install bzr, 
and the qbzr and bzr-git plugins (this might also require bzrtools).  Then, in
a 
git branch, type,

     bzr qlog

... and you'll have a rather beautiful linear history of the master branch,
with 
the option to expand out any individual merge that you want and review the 
commits therein.

It'll take a little longer to load a large history, because it's solving a more 
complex problem than the standard git log tools, but I've found it a rather
nice 
way of simplifying the view of complex, multi-branch history.  I know some 
people have complained about the allegedly tangled mess of dmd/druntime/phobos 
commit history, and this might be a useful trick to help with that.

I'm not aware of any git history visualizer that supports this functionality, 
which is a shame, because it must be possible.
Feb 02 2015
prev sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Monday, February 02, 2015 22:37:13 Joseph Rushton Wakeling via Digitalmars-d
wrote:

 On 02/02/15 15:40, Dicebot via Digitalmars-d wrote:

 Quick summary of my opinion can look like this:

 1) dub is a horrible build tool
 2) git submodules is a horrible way for dependency management

 First thing is something we can improve on in future and, actually, dub
supports
 using rdmd as a build tool already.

 Second thing is an inherent issue that won't change without some fundamental
 breaking changes in how hit submodules are implemented.

 Because of that I am trying to push dub despite the fact I don't like lot of
 things about it personally.


 Very well put.  If there are issues with dub, we can and should address them --
 not turn to a tool that isn't actually intended to solve the problem we are
 trying to address.


I've long thought that it some point, dub would be forced to support
arbitrary build systems in order to actually work in the general case. There
are far too many special cases and more complicated things that build
scripts and the like frequently have to deal with for dub's simple approach
to work in many cases. Just look at what something like cmake it can do. It
blows dub out of the water when it comes to power.

The fact that we have package management via dub is great, and the way it
builds stuff works for many common cases (especially when you're talking
about libraries), but I really don't think that its build system is powerful
enough for the long run. I doubt that Sonke has time to deal with it given
how much is on his plate, but we really need to look at making it so that
dub supports a much more powerful build system - possibly even making it so
that it can support projects building with existing build systems such as
cmake. I'm sure that that sort of thing comes with its own types of
problems, but what dub does right now is just too simplistic.

A prime example that I've wondered about is how to deal with cases where you
need to create a C or C++ wrapper for something and use it in your project.
If dub supported makefiles, then that would be trivial, but as it stands, it
can't do anything even close to that. Anything that's C or C++ has to
already be built no the target system, be in a standard position for the
linker to find it, etc., or you can't use it in a dub project. Given our
need for C/C++ compatability, I think that that's a serious flaw. Dub is
new, and we've gotten by thus far with what it can do, but if it's going to
really be our standard build system for D, it needs to improve by quite a
bit.

- Jonathan M Davis
Feb 02 2015
parent "Mathias LANG" <geod24 gmail.com> writes:
On Monday, 2 February 2015 at 21:55:21 UTC, Jonathan M Davis 
wrote:

 I've long thought that it some point, dub would be forced to 
 support
 arbitrary build systems in order to actually work in the 
 general case. There
 are far too many special cases and more complicated things that 
 build
 scripts and the like frequently have to deal with for dub's 
 simple approach
 to work in many cases. Just look at what something like cmake 
 it can do. It
 blows dub out of the water when it comes to power.

 The fact that we have package management via dub is great, and 
 the way it
 builds stuff works for many common cases (especially when 
 you're talking
 about libraries), but I really don't think that its build 
 system is powerful
 enough for the long run. I doubt that Sonke has time to deal 
 with it given
 how much is on his plate, but we really need to look at making 
 it so that
 dub supports a much more powerful build system - possibly even 
 making it so
 that it can support projects building with existing build 
 systems such as
 cmake. I'm sure that that sort of thing comes with its own 
 types of
 problems, but what dub does right now is just too simplistic.

 A prime example that I've wondered about is how to deal with 
 cases where you
 need to create a C or C++ wrapper for something and use it in 
 your project.
 If dub supported makefiles, then that would be trivial, but as 
 it stands, it
 can't do anything even close to that. Anything that's C or C++ 
 has to
 already be built no the target system, be in a standard 
 position for the
 linker to find it, etc., or you can't use it in a dub project. 
 Given our
 need for C/C++ compatability, I think that that's a serious 
 flaw. Dub is
 new, and we've gotten by thus far with what it can do, but if 
 it's going to
 really be our standard build system for D, it needs to improve 
 by quite a
 bit.

 - Jonathan M Davis


I think it's the moment to point out the latest P.R. merged into 
dub:
https://github.com/D-Programming-Language/dub/pull/489

It does support Makefiles in a way (prebuiltcommand), but I agree 
it's rather simplistic.
Feb 02 2015
prev sibling next sibling parent "Atila Neves" <atila.neves gmail.com> writes:
I have to say that the directory naming issue bit me; I had to 
move all of my code to "source" in order to use dub, and then I 
end up with "cerealed/source/cerealed/*.d" which is kind of silly.

rdmd is awesome, I use dub when the D program I'm writing has dub 
package dependencies. For vibe.d, for instance, I really don't 
want to have to figure out how to build it myself, and rdmd won't 
cut it there, at least not without major contortions.

But both rdmd and dub fall short with anything remotely 
complicated. I've used a lot of CMake features that I simply 
don't have with either of them.

Atila

On Monday, 2 February 2015 at 08:09:39 UTC, Vladimir Panteleev 
wrote:

 On Monday, 2 February 2015 at 05:23:52 UTC, Daniel Murphy wrote:

 "Vladimir Panteleev"  wrote in message 
 news:viqwfixznbdbdwvhavuk forum.dlang.org...


 I don't use Dub


 You really should!  I put it off for months and months but I'm 
 quite happy with it now.


 Even if I had faith that dub was a perfectly polished piece of 
 software, it doesn't solve any problems I have with building D 
 programs, and in fact would make said task more complicated. 
 Here's why.

 1. rdmd

 rdmd is a simple and effective way to build D programs, and I'm 
 sad to see its use declining.

 rdmd leverages two things: D's module system, and the 
 compiler's import search path. Dub's design seems to ignore 
 both of the above.

 1a. rdmd and D's module system:

 When you run `dmd -o- program.d`, the compiler will 
 automatically read all modules imported by your program, and 
 their imports, and so on. It does so by searching the 
 filesystem across its search path for matches which correspond 
 with D's module system, and only reads those files that are 
 needed.

 rdmd leverages this by collecting (and caching) the list of 
 modules used in the program, and passing that list to the 
 compiler. The compiler will then compile no more and no less 
 than the exact set of modules transitively imported by the 
 program.

 In contrast, Dub's default modus operandi is to blindly send to 
 the compiler all *.d files found in the "src" folder, whether 
 they're actually used or not. Not only can this be slower if 
 not all modules are always used, but it also won't work if the 
 source code contains multiple entry points, forcing you to 
 write complicated configuration files (i.e. do the computer's 
 work for it).

 1b. rdmd and D's search path

 rdmd does not have any additional parameters to set up for 
 where it needs to look for source files, because it relies on 
 the compiler's search mechanism. Thus, if you can build your 
 program with rdmd, "dmd -o- program" will succeed, and usually 
 vice versa.

 In contrast, Dub builds its own search path using its JSON 
 configuration files, and has no equivalent of "dmd -o-".

 There is no simple way to syntax-check just one file in a 
 project when using Dub. I rely on this a lot in my workflow - I 
 configured a syntax-check key in my editor, which I use almost 
 as often as I save. A syntax check (dmd -o-) is much faster 
 than a full build, as it skips parsing other parts of the 
 project, code generation, and linking.

 2. git

 I've found that git's submodule feature is an excellent way to 
 manage dependencies across D libraries, and fits really well 
 with D's package system. For clarity to readers not too 
 familiar with Git, I'll explain by example:

 2a. Directory structure

 If you have a library "fruit" with the modules "fruit.apple" 
 and "fruit.banana", create a git repository (in a directory 
 called "fruit") with the files apple.d and banana.d.

 Now, in your project (e.g. "orchard"), add a symlink called 
 "fruit" pointing to the repository:

 ~/
 |- fruit/
 |  |- .git/
 |  |- apple.d
 |  '- banana.d
 '- orchard/
    |- .git/
    |- fruit -> ~/fruit/
    '- tree.d

 tree.d can now import the "fruit.apple" module without any 
 additional compiler switches. `rdmd tree` will just work. This 
 is because the modules of the "fruit" library are placed at the 
 repository root, and the repository's directory name matches 
 with the library's package name, so when the compiler will look 
 for the "fruit.apple" module, it will find it at 
 "fruit/apple.d".

 2b. Distribution

 Upload the "fruit" repository to GitHub. In your "orchard" 
 project, delete the symlink, and run:

 git submodule add https://github.com/You/fruit fruit

 This will clone the repository from GitHub, create a 
 .gitmodules file, and add an entry to the git index. If you 
 commit all these, and put "orchard" on GitHub too, someone can 
 run the command:

 git clone --recursive https://github.com/You/orchard

 ... to get the orchard project, and its dependencies (the fruit 
 library).

 2c. Versioning

 Git stores the exact commit of each submodule in the parent 
 repository.

 So, if you make a commit in orchard/fruit/, and run `git 
 status` in orchard/, git will show you that the "fruit" 
 submodule has been modified.

 Most importantly, this means that any breaking change in the 
 "fruit" library will never affect existing projects which use 
 it, since they will continue to use the registered commit which 
 was known to work.

 This gives you dependency versioning with commit granularity - 
 you can hardly ask for something better - all without messing 
 with configuration files.

 ---

 The one thing I wish git had is symlink support for Windows. 
 Windows has symlinks and directory junctions, but msysgit still 
 doesn't support them.

 For a practical instance of the above example, see Digger:

 https://github.com/CyberShadow/Digger

 Note the simple build instructions:

 https://github.com/CyberShadow/Digger#building

 2d. Git vs. Dub

 Unfortunately, the above-described approach is not compatible 
 with Dub:

 - Dub packages are generally expected to have their source code 
 in a "src" subdirectory, although you can set the source 
 directory to "." in the configuration file.

 - When cloning repositories, dub does not preserve the 
 repository's directory name (so e.g. fruit will be cloned to 
 ~/.dub/fruit-1.0.0/).

 Somebody has created a Dub package for my library (excluding 
 certain packages, due to point 1a above), and the other day 
 someone complained that it doesn't work with Dscanner, because 
 of the above issue - the module path "ae.dir.module" does not 
 correspond to the filesystem path "ae-1.0.1/dir/module.d".

 So, in order to start using Dub, I'd need to:

 - restructure the directory layout of my library (breaking 
 change)
 - update all projects which use this library to use Dub instead
 - give up quick syntax checking
 - give up commit-granularity versioning
 - begin maintaining JSON configuration files
 - begin versioning libraries by hand
 - install Dub on all my computers, servers, and virtual machines

 No thanks.

 I could invest time in improving Dub to fix or ameliorate some 
 of the above points, but I don't see a compelling reason to. In 
 fact, I think we should integrate rdmd into dmd - dmd clearly 
 already knows which source files participate in compilation, as 
 all rdmd does is basically take dmd's output and feed it back 
 to it. This will greatly speed up compilation, too.

 Change my view.
Feb 02 2015
prev sibling next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 02/02/2015 03:09 AM, Vladimir Panteleev wrote:

 1a. rdmd and D's module system:
[...]
 In contrast, Dub's default modus operandi is to blindly send to the
 compiler all *.d files found in the "src" folder, whether they're
 actually used or not. Not only can this be slower if not all modules are
 always used, but it also won't work if the source code contains multiple
 entry points, forcing you to write complicated configuration files (i.e.
 do the computer's work for it).


This is one of my biggest beefs with dub, too, and constantly causes me 
trouble. (I love that dub exists as a package manager, but I REALLY 
dislike that it tries to be build system too. Especially since I don't 
like the way its build functionality works.)

In theory, dub does have the --rdmd switch to make it select source 
files in a sane manner (by deferring to rdmd), but unfortunately it 
doesn't get well-tested and frequently breaks:

https://github.com/D-Programming-Language/dub/issues/492


 1b. rdmd and D's search path

 rdmd does not have any additional parameters to set up for where it
 needs to look for source files, because it relies on the compiler's
 search mechanism. Thus, if you can build your program with rdmd, "dmd
 -o- program" will succeed, and usually vice versa.

 In contrast, Dub builds its own search path using its JSON configuration
 files, and has no equivalent of "dmd -o-".

 There is no simple way to syntax-check just one file in a project when
 using Dub. I rely on this a lot in my workflow - I configured a
 syntax-check key in my editor, which I use almost as often as I save. A
 syntax check (dmd -o-) is much faster than a full build, as it skips
 parsing other parts of the project, code generation, and linking.



-o/-version/etc args to be passed directly into dmd/rdmd/ldmd/gdmd/etc. 
Without that, dub is useless as a MERE package manager. It's either 
packaging AND building, or nothing. (Ever try to use a recent version of 
vibe.d in a project that *doesn't* use dub as its build system? PITA. It 
isn't even recommended to do so.)

I tried to add that feature one time, but I had trouble grokking the 
relevant section of dub's source :(

There's also one other big thing I don't like about it: It needlessly 
reinvents and renames dmd's entire set of command switches. That isn't 
even needed for ldc/gdc anyway since, last I heard, the ldmd and gdmd 
wrappers exist. I pushed to get the *actual* compiler switched accepted 
by dub, and the feature made it in, but trying to use it generates a big 
giant warning for all your library's users complaining that the feature 
*shouldn't* be used. Thus completely defeating the point. Argh.

Again, I love that Dub (and especially it's package repository) exists. 
But dub just tries to hard to impose it's own way.
Feb 02 2015
next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 02/02/2015 05:15 PM, Nick Sabalausky wrote:


 -o/-version/etc args to be passed directly into dmd/rdmd/ldmd/gdmd/etc.


Actually I meant -I, not -o. (But I guess maybe -o too.)
Feb 02 2015
prev sibling next sibling parent =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 02.02.2015 um 23:15 schrieb Nick Sabalausky:

 On 02/02/2015 03:09 AM, Vladimir Panteleev wrote:

 1a. rdmd and D's module system:
 [...]
 In contrast, Dub's default modus operandi is to blindly send to the
 compiler all *.d files found in the "src" folder, whether they're
 actually used or not. Not only can this be slower if not all modules are
 always used, but it also won't work if the source code contains multiple
 entry points, forcing you to write complicated configuration files (i.e.
 do the computer's work for it).


 This is one of my biggest beefs with dub, too, and constantly causes me
 trouble. (I love that dub exists as a package manager, but I REALLY
 dislike that it tries to be build system too. Especially since I don't
 like the way its build functionality works.)


Short answer: This is required to properly support other build systems/IDEs.


 In theory, dub does have the --rdmd switch to make it select source
 files in a sane manner (by deferring to rdmd), but unfortunately it
 doesn't get well-tested and frequently breaks:

 https://github.com/D-Programming-Language/dub/issues/492


Briefly looking at it, I'm pretty sure that this is actually a DMD/RDMD 
issue. The import statements for the internal event driver recently have 
been made function local to speed up compilation. And if nothing has 
changed, DMD simply doesn't output function local dependencies with -o-, 
so that RDMD fails to include the proper modules.


 1b. rdmd and D's search path

 rdmd does not have any additional parameters to set up for where it
 needs to look for source files, because it relies on the compiler's
 search mechanism. Thus, if you can build your program with rdmd, "dmd
 -o- program" will succeed, and usually vice versa.

 In contrast, Dub builds its own search path using its JSON configuration
 files, and has no equivalent of "dmd -o-".

 There is no simple way to syntax-check just one file in a project when
 using Dub. I rely on this a lot in my workflow - I configured a
 syntax-check key in my editor, which I use almost as often as I save. A
 syntax check (dmd -o-) is much faster than a full build, as it skips
 parsing other parts of the project, code generation, and linking.



 -o/-version/etc args to be passed directly into dmd/rdmd/ldmd/gdmd/etc.
 Without that, dub is useless as a MERE package manager. It's either
 packaging AND building, or nothing. (Ever try to use a recent version of
 vibe.d in a project that *doesn't* use dub as its build system? PITA. It
 isn't even recommended to do so.)


There is "dub describe" for this. There is also a pending feature to 
support the use of shell variables instead of piping everything to stdout.


 I tried to add that feature one time, but I had trouble grokking the
 relevant section of dub's source :(

 There's also one other big thing I don't like about it: It needlessly
 reinvents and renames dmd's entire set of command switches. That isn't
 even needed for ldc/gdc anyway since, last I heard, the ldmd and gdmd
 wrappers exist. I pushed to get the *actual* compiler switched accepted
 by dub, and the feature made it in, but trying to use it generates a big
 giant warning for all your library's users complaining that the feature
 *shouldn't* be used. Thus completely defeating the point. Argh.


The question is how tight everything should be bound to DMD. For 
example, does SDC have an sdmd wrapper? The goal was to make the package 
description independent of the compiler used to build it (at least in 
most cases). An alternative way to using an own way to describe build 
options would of course have been to always rely on DMD's option format 
and translate as required (which is supported, as you already 
mentioned). But that will in turn feel strange to anyone not using DMD.


 Again, I love that Dub (and especially it's package repository) exists.
 But dub just tries to hard to impose it's own way.
Feb 05 2015
prev sibling parent Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 2 February 2015 at 22:15, Nick Sabalausky via Digitalmars-d

 There's also one other big thing I don't like about it: It needlessly
 reinvents and renames dmd's entire set of command switches. That isn't even
 needed for ldc/gdc anyway since, last I heard, the ldmd and gdmd wrappers
exist.


Just because they exist does not mean they are being used.  (ie: gdmd
has been split away and EOL for over a year now).
Feb 05 2015
prev sibling next sibling parent reply "ponce" <contact gam3sfrommars.fr> writes:
So. I've reported lots of bugs in DUB and fixed some, and since 
using it I never looked back. It does seem more stable these days.

On Monday, 2 February 2015 at 08:09:39 UTC, Vladimir Panteleev 
wrote:

 So, in order to start using Dub, I'd need to:

 - restructure the directory layout of my library (breaking 
 change)


I was about to say "yes" but perhaps that's fixable in DUB.
Currently, using ae checkouted in an ae/ directory means having 
an "import leak" meaning that you have to import a path outside 
the checkouted repositery. That's somehow unhygienic like, er, a 
global namespace. But yeah DUB could isolate checkouts one 
directory deeper (ae-1.0.0/ae/) to make this a non-problem, this 
would support libraries like ae/ and make DScanner work.


 - update all projects which use this library to use Dub instead


And drop their current build system for something hopefully more 
uniform and not that complicated.


 - give up quick syntax checking


Fixed it you push code one directory further.


 - give up commit-granularity versioning


Not necessarily if you make tags at each commit (and most of the 
time it's not really necessary).


 - begin maintaining JSON configuration files


It's not that bad honestly. Actually mandatory field are few and 
between. If using sub-packages it does take some learning though.


 - begin versioning libraries by hand


Yes, but with the benefit of:
- helping your users choosing a version, and protecting them from 
churn (eg. broken build)
- choosing if they want breaking changes.
- see also: 
http://forum.dlang.org/post/nknzdktahezyztzdvwpz forum.dlang.org 
for the diamond dependency problem.

git submodules work well when you don't use third-party libs but 
let's be honest, not everyone understand git submodules let alone 
want to. I also know some D programmers that don't want to touch 
git.


 No thanks.

 I could invest time in improving Dub to fix or ameliorate some 
 of the above points, but I don't see a compelling reason to. In 
 fact, I think we should integrate rdmd into dmd - dmd clearly 
 already knows which source files participate in compilation, as 
 all rdmd does is basically take dmd's output and feed it back 
 to it. This will greatly speed up compilation, too.

 Change my view.


The DUB way does take more work and requires some learning but it 
makes things super simple for library _users_ and imho this 
benefits the ecosystem as a whole. If you want more users then 
DUB is nicer to them.

I've also found it changed my way to develop, I'm much more 
likely to reuse something from the registry, and much more likely 
to bundle breaking changes together.
Feb 02 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 2 February 2015 at 22:42:35 UTC, ponce wrote:

 - choosing if they want breaking changes.


Currently I have a super-simple way to deal with breaking changes:

I prefix the commit message with "[BREAKING] ".

Not perfect, but it makes tracking them down much easier in the 
history.
Feb 02 2015
parent "ponce" <contact gam3sfrommars.fr> writes:
On Tuesday, 3 February 2015 at 02:35:50 UTC, Vladimir Panteleev 
wrote:

 On Monday, 2 February 2015 at 22:42:35 UTC, ponce wrote:

 - choosing if they want breaking changes.


 Currently I have a super-simple way to deal with breaking 
 changes:

 I prefix the commit message with "[BREAKING] ".

 Not perfect, but it makes tracking them down much easier in the 
 history.


I do this too. SemVer doesn't beat reading commit logs for 
accuracy.
Feb 03 2015
prev sibling next sibling parent "weaselcat" <weaselcat gmail.com> writes:
On Monday, 2 February 2015 at 08:09:39 UTC, Vladimir Panteleev 
wrote:

 ...


Sorry if it has been answered, but one of the points brought up 
was non-D project management. Is it possible for Dub to i.e, 
compile C files for me and link them with my D files? I'm 
currently using makefiles for this.
Feb 02 2015
prev sibling next sibling parent reply "Martin Nowak" <code dawg.eu> writes:
 - restructure the directory layout of my library (breaking 
 change)


That's likely solveable. Haven't seen anyone putting sources in 
the root dir for ages though, mostly because it contains readmes 
and configs.


 - update all projects which use this library to use Dub instead


If we can solve the root Dir issue, then you can use both in 
parallel.


 - give up quick syntax checking


What keeps you from calling DMD?


 - give up commit-granularity versioning


You should be careful with breaking changes, then the granularity 
isn't needed.


 - begin maintaining JSON configuration files


It's not zero configuration, but we recently pimped the init 
command to take dependencies. And there is going to be SDL as 
alternative to JSON.


 - begin versioning libraries by hand


Semantic versioning is a good thing, because it allows sensible 
dependency updates.
You also need library versions for your changlog and issues.


 - install Dub on all my computers, servers, and virtual machines


We'll distribute it with DMD soon.

 No thanks.

 I could invest time in improving Dub to fix or ameliorate some 
 of the above points, but I don't see a compelling reason to. In 
 fact, I think we should integrate rdmd into dmd - dmd clearly 
 already knows which source files participate in compilation, as 
 all rdmd does is basically take dmd's output and feed it back 
 to it. This will greatly speed up compilation, too.


We could also add a lot of the rdmd workflow to dub.
https://github.com/D-Programming-Language/dub/issues/103

There seems to be a general scepticism against dub and I wonder 
what the reasons are.
A lack of good documentation and guides is clearly the main 
hurdle for starters, but there seems to be more, so I'm glad that 
you shared this.
Feb 02 2015
next sibling parent "Tofu Ninja" <emmons0 purdue.edu> writes:
On Tuesday, 3 February 2015 at 02:19:56 UTC, Martin Nowak wrote:

 There seems to be a general scepticism against dub and I wonder 
 what the reasons are.


Personally I love it and have had no problems with it, so yeah...
Feb 02 2015
prev sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Tue, 03 Feb 2015 02:19:55 +0000, Martin Nowak wrote:


 There seems to be a general scepticism against dub and I wonder what the
 reasons are.


'cause it really sux as a build tool.=
Feb 03 2015
next sibling parent reply Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 2015-02-03 at 08:51 +0000, ketmar via Digitalmars-d wrote:

 On Tue, 03 Feb 2015 02:19:55 +0000, Martin Nowak wrote:
 

 There seems to be a general scepticism against dub and I wonder what the
 reasons are.


 
 'cause it really sux as a build tool.


I am reminded of some history that this thread has prompted me to write:

I really dislike XML as a human authored format. In 2006 I got so angry
with Ant, I wrote Gant as a way of scripting Ant tasks – the way people
were using XML to create imperative builds was getting just too awful.
Maven has declarative project specification, but if you actually want to
build anything you have to write things in Java. Gant got (friendly)
forked and is now officially the Groovy front end to Ant. Not many users
though. Nor for Gant itself really.

In 2007, Hans Dockter and I tried to use Gant as the basis of a build
framework. It was able to do the job, but it was awkward and we declared
it not a good direction. Hans started from scratch with all the lessons
learned from Gant, Ant and Maven, and with Adam Murdoch, and occasional
input from me, created Gradle. Not quite the Gradle of today, but the
beginnings of what is there today.

Go folk, and I suspect many D folk, will not want to use a JVM-based
build framework even though it has C++ as well as support for JVM
languages.

Clearly there is appetite for a dependency and build framework for D/C
++/C which is fine, Gradle could do it, SCons and CMake probably not.

Many are liking Dub, many are not. Myself, I have doubts, but I haven't
really got stuck in. Perhaps we can surmise that Dub points at a very
good idea for D/C++/C systems, but is not itself it. Mayhap there needs
to be a "child of Dub".

The issue is though that Dub was created because someone did something
rather than just talking (and emailing) about it. As Hans and colleagues
took knowledge and experience and created Gradle, somebody, or group of
people, need to take the practical experience of Make, CMake, Jam,
SCons, Waf, Ant, Gant, Maven, Gradle and Dub, and do something.

I will also say that Bintray and Artifactory may be better
artefact/dependency stores that trying to build something D-specific
from scratch.


-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Feb 03 2015
next sibling parent reply "Paolo Invernizzi" <paolo.invernizzi no.address> writes:
On Tuesday, 3 February 2015 at 17:34:05 UTC, Russel Winder wrote:

 The issue is though that Dub was created because someone did 
 something
 rather than just talking (and emailing) about it. As Hans and 
 colleagues
 took knowledge and experience and created Gradle, somebody, or 
 group of
 people, need to take the practical experience of Make, CMake, 
 Jam,
 SCons, Waf, Ant, Gant, Maven, Gradle and Dub, and do something.

 I will also say that Bintray and Artifactory may be better
 artefact/dependency stores that trying to build something 
 D-specific
 from scratch.


Don't forget tup!!! [1]

[1] http://gittup.org/tup/
Feb 03 2015
next sibling parent reply "Jonathan Marler" <johnnymarler gmail.com> writes:
On Tuesday, 3 February 2015 at 20:18:31 UTC, Paolo Invernizzi 
wrote:

 [1] http://gittup.org/tup/


I don't know who wrote that site but they are quite hilarious :)

"Unfortunately, tup is so fast that your chair mounted jousting 
might suffer. I apologize in advance if someone besmirches your 
honor and you are unable to properly defend yourself as a result."

"In tup, the arrows go up. This is obviously true because it 
rhymes. "
Feb 03 2015
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, Feb 03, 2015 at 08:41:38PM +0000, Jonathan Marler via Digitalmars-d
wrote:

 On Tuesday, 3 February 2015 at 20:18:31 UTC, Paolo Invernizzi wrote:

[1] http://gittup.org/tup/


 
 I don't know who wrote that site but they are quite hilarious :)
 
 "Unfortunately, tup is so fast that your chair mounted jousting might
 suffer. I apologize in advance if someone besmirches your honor and
 you are unable to properly defend yourself as a result."
 
 "In tup, the arrows go up. This is obviously true because it rhymes. "


Don't be fooled, though. Tup contains some revolutionary ideas that
would-be implementors of the Next Great Build System would do well to
take note.  The paper linked to by the website explains the difference
between "first generation" build algorithms vs. "second generation"
build algorithms.

In brief, first generation build algorithms are centered around linearly
scanning dependencies to update, which is not scalable, because it is
O(n), and as the size of the source tree grows large, linear scanning
becomes prohibitively expensive. Your code-compile-test cycle gets
bottlenecked at the compile step because every single time the poor
build tool has to scan the entire source tree of hundreds or even
thousands of source files, and rebuilding the graph from scratch.
Furthermore, any attempt to reduce the cost of the linear scan (e.g.,
invoke make from a subdirectory) breaks build correctness, because the
algorithm is unable to reliably determine the smallest consistent
subgraph that must be updated when some node(s) change unless it scans
everything.

Second generation build algorithms are centered around *not* scanning,
but taking advantage of modern OSes providing APIs for file change
notification. Rather than scan the whole source tree every time, it
takes the changeset as input -- either from the OS, or from some other
source of information. By leveraging OS features, we can obtain this
info on an as-needed basis instead of an O(n) scan. Upon this, a
modified algorithm with O(log n) complexity is built, that at the same
time also guarantees correctness -- by the modified representation of
the dependency graph, it is possible to derive the smallest consistent
subgraph that needs to be updated when some changes are made.

In addition to an overhaul of the core algorithm, tup also has some
clever tricks, like wrapping libc of executed commands, so that all
*actual* inputs to the compiler (and other build commands) are included
in the dependency graph -- if the compiler reads /usr/include/stdio.h,
for example, tup will know that the source file depends on stdio.h --
and this *without* needing to scan the source file manually or asking
the user to specify such tedious things.

In fact, tup is just a proof-of-concept tool (that's surprisingly usable
for what it is!); the author expects that build tools of the future will
develop its ideas further. As for me, I get the feeling that tup has the
right ideas, even if it isn't quite "it" yet -- the implementation isn't
quite what I would've chosen. But nevertheless, it holds a lot of
promise for the future, unlike most other alternative build tools, which
are just more-of-the-same-rehashed-the-nth-time.


T

-- 
What are you when you run out of Monet? Baroque.
Feb 03 2015
next sibling parent reply Martin Nowak <code+news.digitalmars dawg.eu> writes:
On 02/03/2015 10:02 PM, H. S. Teoh via Digitalmars-d wrote:

 Rather than scan the whole source tree every time, it
 takes the changeset as input -- either from the OS, or from some other
 source of information.


Indeed a good idea, although according to his graphs, this only starts 
to matter for +1000 files.
Feb 03 2015
next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Feb 04, 2015 at 02:30:23AM +0100, Martin Nowak via Digitalmars-d wrote:

 On 02/03/2015 10:02 PM, H. S. Teoh via Digitalmars-d wrote:

Rather than scan the whole source tree every time, it takes the
changeset as input -- either from the OS, or from some other source
of information.


 
 Indeed a good idea, although according to his graphs, this only starts
 to matter for +1000 files.


The size of software projects these days tend to dwarf that easily.  For
a personal website project of mine, for example, SCons is taking
noticeably long just to scan the entire directory tree for changes. (But
then again, I do have a *lot* of intermediate build targets.) If a
one-man project is already running into these performance limits, how
much faster enterprise projects?


T

-- 
Beware of bugs in the above code; I have only proved it correct, not tried it.
-- Donald Knuth
Feb 03 2015
prev sibling parent "Atila Neves" <atila.neves gmail.com> writes:
On Wednesday, 4 February 2015 at 01:30:41 UTC, Martin Nowak wrote:

 On 02/03/2015 10:02 PM, H. S. Teoh via Digitalmars-d wrote:

 Rather than scan the whole source tree every time, it
 takes the changeset as input -- either from the OS, or from 
 some other
 source of information.


 Indeed a good idea, although according to his graphs, this only 
 starts to matter for +1000 files.


Which is a pittance for some projects. In my previous job, the 
Makefiles generated by CMake were taking 8s for a no-op build, 
which was far too long for me. That's how I discovered Ninja. It 
promptly went down to 1s. But we're talking about a 100k SLOC 
project here, most personal repos won't see the difference. 
Almost all professional ones will.

Atila
Feb 04 2015
prev sibling next sibling parent reply "Paolo Invernizzi" <paolo.invernizzi no.address> writes:
On Tuesday, 3 February 2015 at 21:04:46 UTC, H. S. Teoh wrote:

 In addition to an overhaul of the core algorithm, tup also has 
 some
 clever tricks, like wrapping libc of executed commands, so that 
 all
 *actual* inputs to the compiler (and other build commands) are 
 included
 in the dependency graph -- if the compiler reads 
 /usr/include/stdio.h,
 for example, tup will know that the source file depends on 
 stdio.h --
 and this *without* needing to scan the source file manually or 
 asking
 the user to specify such tedious things.


There's more, using this trick, it also check the output of the 
compiler: orphans or unspecified targets are not allowed.
In this way, it has the complete graph of what is generated by 
the build process, and, literally, there's no need for a clean, 
ever.

If you modify a target file, tup will warn you that it will be 
overwritten; if you add a build rule that will output a new file, 
overwriting something that already exists, it will refuse to run, 
and will ask you to remove the current file, and so on...

We are using it a lot, here at work, and we are really enjoying 
it: we've a D tool that emits tup rules, for the chronicle... ;-)
---
P
Feb 04 2015
parent "Fool" <fool dlang.org> writes:
I had a look at different build systems, recently. Tup certainly 
has some progressive ideas. What I don't like is that it 
patronizes its users. I don't believe that tracking all file 
reads and writes is always a good choice. It might be a good 
default, though.

Some other interesting build systems I came accross (in 
alphabetical order):

  - http://www.fastbuild.org
  - http://jpakkane.github.io/meson/
  - http://shakebuild.com
Feb 04 2015
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-02-03 22:02, H. S. Teoh via Digitalmars-d wrote:


 Second generation build algorithms are centered around *not* scanning,
 but taking advantage of modern OSes providing APIs for file change
 notification. Rather than scan the whole source tree every time, it
 takes the changeset as input -- either from the OS, or from some other
 source of information. By leveraging OS features, we can obtain this
 info on an as-needed basis instead of an O(n) scan.


Does this require some kind of daemon running in the background?

-- 
/Jacob Carlborg
Feb 08 2015
parent "Paolo Invernizzi" <paolo.invernizzi no.address> writes:
On Sunday, 8 February 2015 at 20:34:19 UTC, Jacob Carlborg wrote:

 On 2015-02-03 22:02, H. S. Teoh via Digitalmars-d wrote:


 Second generation build algorithms are centered around *not* 
 scanning,
 but taking advantage of modern OSes providing APIs for file 
 change
 notification. Rather than scan the whole source tree every 
 time, it
 takes the changeset as input -- either from the OS, or from 
 some other
 source of information. By leveraging OS features, we can 
 obtain this
 info on an as-needed basis instead of an O(n) scan.


 Does this require some kind of daemon running in the background?


Yes, but... practically nobody uses the daemon (which is 
optional).
There's practically no difference in performance against a simple 
scan of the FS.

---
Paolo
Feb 09 2015
prev sibling next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Tuesday, 3 February 2015 at 20:18:31 UTC, Paolo Invernizzi 
wrote:

 Don't forget tup!!! [1]

 [1] http://gittup.org/tup/


This is first time I hear about tup but I must admit it looks 
really cool. It has all the good thing I love about make but with 
more thinking being put into it and no awkward legacy.

And it is honest natively compiled program with no extra annoying 
dependencies.

It would tempting to include something like that in Phobos as a 
library - sadly, it is licenced under GPL and thus this option is 
not available :(
Feb 03 2015
parent "Jonathan Marler" <johnnymarler gmail.com> writes:
On Tuesday, 3 February 2015 at 20:45:03 UTC, Dicebot wrote:

 On Tuesday, 3 February 2015 at 20:18:31 UTC, Paolo Invernizzi 
 wrote:

 Don't forget tup!!! [1]

 [1] http://gittup.org/tup/


 This is first time I hear about tup but I must admit it looks 
 really cool. It has all the good thing I love about make but 
 with more thinking being put into it and no awkward legacy.

 And it is honest natively compiled program with no extra 
 annoying dependencies.

 It would tempting to include something like that in Phobos as a 
 library - sadly, it is licenced under GPL and thus this option 
 is not available :(


I'm reading the paper on it now, looks very interesting.  Tup may 
be licensed but I'm sure we could incorporate the algorithms it 
uses into phobos somehow...I'll know more after I finish reading.
Feb 03 2015
prev sibling next sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
On Tuesday, 3 February 2015 at 20:18:31 UTC, Paolo Invernizzi 
wrote:

 On Tuesday, 3 February 2015 at 17:34:05 UTC, Russel Winder 
 wrote:

 The issue is though that Dub was created because someone did 
 something
 rather than just talking (and emailing) about it. As Hans and 
 colleagues
 took knowledge and experience and created Gradle, somebody, or 
 group of
 people, need to take the practical experience of Make, CMake, 
 Jam,
 SCons, Waf, Ant, Gant, Maven, Gradle and Dub, and do something.

 I will also say that Bintray and Artifactory may be better
 artefact/dependency stores that trying to build something 
 D-specific
 from scratch.


 Don't forget tup!!! [1]

 [1] http://gittup.org/tup/


I didn't forget it myself, but thanks for bringing it up. I'm 
considering having a tup backend actually.
Tup is cool but unfortunately is still "too manual", for lack of 
a better term. I'd much rather write "add_executable" and not 
worry about the details than write the commands explicitly 
myself. But it's certainly interesting as an alternative backend 
to, say, Ninja.

Atila
Feb 03 2015
parent reply "Dicebot" <public dicebot.lv> writes:
On Tuesday, 3 February 2015 at 21:05:38 UTC, Atila Neves wrote:

 I didn't forget it myself, but thanks for bringing it up. I'm 
 considering having a tup backend actually.
 Tup is cool but unfortunately is still "too manual", for lack 
 of a better term. I'd much rather write "add_executable" and 
 not worry about the details than write the commands explicitly 
 myself. But it's certainly interesting as an alternative 
 backend to, say, Ninja.


I would warn against this attitude. Trying to do too much magic 
is one of reasons I ignore all of modern build tools and still 
keep my makefiles. There is huge benefit in knowing that your 
build tool can express any dependency tree based workflow in 
uniform manner - be it compiling sometithing, downloading remote 
artifacts or generating a package.

With a good base "smart" solutions can be built on top. This is 
actually how we use make in Sociomantic - by having a set of 
standard makefiles with D-specific rules that allow to define 
build target as simple as this:

$B/appname: $C/src/appname/main.d

all += $B/appname

(yes, that is all that needs to be in actual makefile)
Feb 03 2015
parent "Atila Neves" <atila.neves gmail.com> writes:
 I would warn against this attitude. Trying to do too much magic 
 is one of reasons I ignore all of modern build tools and still 
 keep my makefiles. There is huge benefit in knowing that your 
 build tool can express any dependency tree based workflow in 
 uniform manner - be it compiling sometithing, downloading 
 remote artifacts or generating a package.


They're not mutually exclusive. High-level convenience can 
happily coexist with low-level control. Isn't that why we're on 
this forum? ;) Once more, CMake handles this well, modulo the 
terrible language to do it in.


 With a good base "smart" solutions can be built on top. This is 
 actually how we use make in Sociomantic - by having a set of 
 standard makefiles with D-specific rules that allow to define 
 build target as simple as this:

 $B/appname: $C/src/appname/main.d

 all += $B/appname

 (yes, that is all that needs to be in actual makefile)


That's pretty cool. I'm aiming for something along those lines.

Atila
Feb 04 2015
prev sibling parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 2015-02-03 at 20:18 +0000, Paolo Invernizzi via Digitalmars-d wrote:

 


[…]

 Don't forget tup!!! [1]
 
 [1] http://gittup.org/tup/


You are quite right to point out I had missed that. I haven't had time 
to use it properly as yet, but I am following development. I really 
should Just Use It™

Whilst we are raising missing things, there are two historical things 
that would be worth at least looking at because they had things to 
contribute that appear to have been lost (*).

Aegis was (actually still is, but…) a distributed configuration 
management system which could be used for source code version control 
or dependency management. It could have taken the world by storm, but 
everyone decided Subversion was The Thing™ 
http://aegis.sourceforge.net/

Vesta was (still is, but only sort of) a configuration management 
system which managed source and compilation products on a global 
scale. Compile stuff at one site and source and compilation product 
becomes available globally to anyone who wants it. 
http://vesta.sourceforge.net/

Whilst talking this sort of thing I should perhaps mention Fossil, 
which makes Git look awkward. In a sense it is sad that the world 
assumes that if it is FOSS it is on GitHub. Especially if you are 
Atlassian and you bought BitBucket.


(*) At least temporarily in the way that actors, dataflow, CSP got 
totally lost in 1990s and 2000s.

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Feb 04 2015
prev sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Tue, 03 Feb 2015 17:33:53 +0000, Russel Winder via Digitalmars-d wrote:


 The issue is though that Dub was created because someone did something
 rather than just talking (and emailing) about it.


i have my build system with automatic dependency tracking and flexible=20
scripting for many years now. adding D support for it as a task of one=20
day. now i have my own repository of packages in "~/.D" (symlinks,=20
actually), and can use things like "dlang.require 'derelict.sdl2' ;",=20
which will add both Derelict.SDL2 and Derelict.Util as dependency. it's=20
in no way ready for public usage, but it works for me perfectly.

ah, and my build system still supports C/C++/ObjC, so i can freely mix=20
libraries. and i can pass USE_DMD=3D1 flag to use DMD instead of GDC.

so why should i try to invest my time in writing yet another build tool=20
on D, when my own works like a charm? and it can even do various checking=20
a-la 'configure'. and hey, i have "99 bottles of beer" written for it!

yet when there are threads about dub, i'm talking about the things dub=20
need, got either no answer at all or "use external script/no", and then=20
see that dub still sux.=
Feb 03 2015
parent reply "eles" <eles eles.com> writes:
On Wednesday, 4 February 2015 at 05:35:44 UTC, ketmar wrote:

 On Tue, 03 Feb 2015 17:33:53 +0000, Russel Winder via 
 Digitalmars-d wrote:



 in no way ready for public usage.


In the context of this discussion, if you can spare 5 or 10 
minutes, it could be useful if you could provide a more detalied 
description of the tool and the constraints/choices that you had 
to do.

I kinda was always looking for the perfect build system and 
messed with make, cmake, scons, waf, fbuild 
(https://github.com/felix-lang/fbuild), boost.build, biicode etc.

Yes, not much ant, maven or gradle.
Feb 04 2015
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Wed, 04 Feb 2015 09:26:37 +0000, eles wrote:


 On Wednesday, 4 February 2015 at 05:35:44 UTC, ketmar wrote:

 On Tue, 03 Feb 2015 17:33:53 +0000, Russel Winder via Digitalmars-d
 wrote:


=20

 in no way ready for public usage.


=20
 In the context of this discussion, if you can spare 5 or 10 minutes, it
 could be useful if you could provide a more detalied description of the
 tool and the constraints/choices that you had to do.


actually, i'm using a heavily modified fork of jam[1]. i don't want to=20
advertise it, though, 'cause it's very "ketmar-specific", and i have no=20
plans to make it usable for anyone who has some different views on build=20
process. ;-)


 I kinda was always looking for the perfect build system and messed with
 make, cmake, scons, waf, fbuild (https://github.com/felix-lang/fbuild),
 boost.build, biicode etc.
=20
 Yes, not much ant, maven or gradle.


you can look at it here[2], but beware! it's not very well documented,=20
it's incompatible with any other jam fork out here, and it doesn't=20
contain ALL changes. ;-)

i was looking for good build system too, and jam was the one that=20
*almost* fits my needs at the time. then i started to fix some things=20
there, adding some things, and so on. i was never trying to write yet=20
another "universal build system for anyone", it's just a tool for me and=20
my friends (yes, it actually has other users except me! ;-).

i dropped windows support some time ago (just don't need that, 'cause i'm=20
doing cross-compiles from GNU/Linux anyway) and added 'k8jam config'=20
supporting scripts, which does some checks that 'configure' does. it may=20
work on non GNU/Linux OSes, but i never check that ('cause i don't need=20
that ;-).


[1] http://en.wikipedia.org/wiki/Perforce_Jam
[2] http://repo.or.cz/w/k8jam.git
=
Feb 04 2015
parent reply "eles" <eles eles.com> writes:
On Wednesday, 4 February 2015 at 09:44:57 UTC, ketmar wrote:

 On Wed, 04 Feb 2015 09:26:37 +0000, eles wrote:


 On Wednesday, 4 February 2015 at 05:35:44 UTC, ketmar wrote:

 On Tue, 03 Feb 2015 17:33:53 +0000, Russel Winder via 
 Digitalmars-d







 actually, i'm using a heavily modified fork of jam[1]. i don't



 you can look at it here[2], but beware! it's not very well


Hey, many thanks! Return of real experience is always a very 
valuable. I will have a look at it this week-end.
Feb 04 2015
parent ketmar <ketmar ketmar.no-ip.org> writes:
On Wed, 04 Feb 2015 10:11:02 +0000, eles wrote:


 On Wednesday, 4 February 2015 at 09:44:57 UTC, ketmar wrote:

 On Wed, 04 Feb 2015 09:26:37 +0000, eles wrote:


 On Wednesday, 4 February 2015 at 05:35:44 UTC, ketmar wrote:

 On Tue, 03 Feb 2015 17:33:53 +0000, Russel Winder via Digitalmars-d






=20

 actually, i'm using a heavily modified fork of jam[1]. i don't


=20

 you can look at it here[2], but beware! it's not very well


=20
 Hey, many thanks! Return of real experience is always a very valuable. I
 will have a look at it this week-end.


feel free to mail me if you have any questions. i'm not giving any=20
guarantees, but i'll try to answer as good as i can. ;-)=
Feb 04 2015
prev sibling next sibling parent reply Martin Nowak <code+news.digitalmars dawg.eu> writes:
On 02/03/2015 09:51 AM, ketmar wrote:

 'cause it really sux as a build tool.


Not getting into any of the lengthy discussions of yours, but 'it sux' 
isn't really helping anyone to improve it.
Feb 03 2015
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Tue, 03 Feb 2015 19:43:53 +0100, Martin Nowak wrote:


 On 02/03/2015 09:51 AM, ketmar wrote:

 'cause it really sux as a build tool.


=20
 Not getting into any of the lengthy discussions of yours, but 'it sux'
 isn't really helping anyone to improve it.


repeating the reasons why it sux doesn't help too. this thread alone has=20
some of them, including inability to do separate compilation (and the=20
reason why this can be needed). choosing to answer to my "sux" message=20
instead of that will surely help alot.=
Feb 03 2015
parent reply =?ISO-8859-15?Q?S=F6nke_Ludwig?= <sludwig rejectedsoftware.com> writes:
Am 03.02.2015 um 21:08 schrieb ketmar:

 On Tue, 03 Feb 2015 19:43:53 +0100, Martin Nowak wrote:


 On 02/03/2015 09:51 AM, ketmar wrote:

 'cause it really sux as a build tool.


 Not getting into any of the lengthy discussions of yours, but 'it sux'
 isn't really helping anyone to improve it.


 repeating the reasons why it sux doesn't help too. this thread alone has
 some of them, including inability to do separate compilation (and the
 reason why this can be needed).


Separate per package (the default)? Separate by D module 
(--build-mode=singleFile)? Something else?
Feb 05 2015
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Thu, 05 Feb 2015 14:48:12 +0100, S=C3=B6nke Ludwig wrote:


 Am 03.02.2015 um 21:08 schrieb ketmar:

 On Tue, 03 Feb 2015 19:43:53 +0100, Martin Nowak wrote:


 On 02/03/2015 09:51 AM, ketmar wrote:

 'cause it really sux as a build tool.


 Not getting into any of the lengthy discussions of yours, but 'it sux'
 isn't really helping anyone to improve it.


 repeating the reasons why it sux doesn't help too. this thread alone
 has some of them, including inability to do separate compilation (and
 the reason why this can be needed).


=20
 Separate per package (the default)? Separate by D module
 (--build-mode=3DsingleFile)? Something else?


yes, incremental rebuilds, please. even for simple project with two=20
modules dub is not able to do incremental rebuilds:

=3D=3D=3D source/app.d =3D=3D=3D
import std.stdio;
import mymod;

void main () {
  writeln(test);
}

=3D=3D=3D source/mymod.d =3D=3D=3D
module mymod;

int test () { return 42; }

% dub --build-mode=3DsingleFile
Building _00 ~master configuration "application", build type debug.
Compiling using dmd...
Compiling source/app.d...
Compiling source/mymod.d...
Linking...
Running ./_00
42

so far, so good.

touch source/app.d
% dub --build-mode=3DsingleFile
Building _00 ~master configuration "application", build type debug.
Compiling using dmd...
Compiling source/app.d...
Compiling source/mymod.d...
Linking...
Running ./_00
42

??? i didn't modified "mymod.d"! why it is rebuilding it?! 'cmon, it's in=20
no way better than a simple shell script that just executes "dmd -c" for=20
each file and then links the results.=
Feb 05 2015
next sibling parent reply =?ISO-8859-15?Q?S=F6nke_Ludwig?= <sludwig rejectedsoftware.com> writes:
Am 05.02.2015 um 16:01 schrieb ketmar:

 On Thu, 05 Feb 2015 14:48:12 +0100, Sönke Ludwig wrote:


 Am 03.02.2015 um 21:08 schrieb ketmar:

 On Tue, 03 Feb 2015 19:43:53 +0100, Martin Nowak wrote:


 On 02/03/2015 09:51 AM, ketmar wrote:

 'cause it really sux as a build tool.


 Not getting into any of the lengthy discussions of yours, but 'it sux'
 isn't really helping anyone to improve it.


 repeating the reasons why it sux doesn't help too. this thread alone
 has some of them, including inability to do separate compilation (and
 the reason why this can be needed).


 Separate per package (the default)? Separate by D module
 (--build-mode=singleFile)? Something else?


 yes, incremental rebuilds, please. even for simple project with two
 modules dub is not able to do incremental rebuilds:

 === source/app.d ===
 import std.stdio;
 import mymod;

 void main () {
    writeln(test);
 }

 === source/mymod.d ===
 module mymod;

 int test () { return 42; }

 % dub --build-mode=singleFile
 Building _00 ~master configuration "application", build type debug.
 Compiling using dmd...
 Compiling source/app.d...
 Compiling source/mymod.d...
 Linking...
 Running ./_00
 42

 so far, so good.

 touch source/app.d
 % dub --build-mode=singleFile
 Building _00 ~master configuration "application", build type debug.
 Compiling using dmd...
 Compiling source/app.d...
 Compiling source/mymod.d...
 Linking...
 Running ./_00
 42

 ??? i didn't modified "mymod.d"! why it is rebuilding it?! 'cmon, it's in
 no way better than a simple shell script that just executes "dmd -c" for
 each file and then links the results.


Yes, incremental building is indeed a missing feature. It's simply a 
matter of available developer time, as for many of the other concerns. 
Otherwise this is something that has been acknowledged for inclusion 
basically since the beginning of the project.
Feb 05 2015
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Thu, 05 Feb 2015 16:05:11 +0100, S=C3=B6nke Ludwig wrote:


 Yes, incremental building is indeed a missing feature. It's simply a
 matter of available developer time, as for many of the other concerns.
 Otherwise this is something that has been acknowledged for inclusion
 basically since the beginning of the project.


alas, without this feature dub is not a build system. i understand that=20
there are alot of things to do and the resources are limited. that's why=20
i think that dub should be separated to two projects: package manager and=20
build system.

the package manager part should have good "machine" interface to allow=20
it's usage in various scripts and other build systems. and build=20
system... oh, well, it can simply use package manager. this way we can=20
discuss "dub-pkg" and "dub-build" as separate entities, and flaws of one=20
project will not automatically propagate on another project.=
Feb 05 2015
next sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
 the package manager part should have good "machine" interface 
 to allow
 it's usage in various scripts and other build systems. and build
 system... oh, well, it can simply use package manager. this way 
 we can
 discuss "dub-pkg" and "dub-build" as separate entities, and 
 flaws of one
 project will not automatically propagate on another project.


The machine interface is "dub describe". It's JSON, so... that's 
what my Emacs package uses to pass -I flags to dmd for on-the-fly 
syntax checks.

Atila
Feb 05 2015
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Thu, 05 Feb 2015 15:21:08 +0000, Atila Neves wrote:


 the package manager part should have good "machine" interface to allow
 it's usage in various scripts and other build systems. and build
 system... oh, well, it can simply use package manager. this way we can
 discuss "dub-pkg" and "dub-build" as separate entities, and flaws of
 one project will not automatically propagate on another project.


=20
 The machine interface is "dub describe". It's JSON, so... that's what my
 Emacs package uses to pass -I flags to dmd for on-the-fly syntax checks.


there is too much noise there. and there is no easy access to package=20
manager itself. as far as i can see, there is no command even to list all=20
*known* packages, only installed ones.=
Feb 05 2015
next sibling parent reply "Mathias LANG" <geod24 gmail.com> writes:
On Thursday, 5 February 2015 at 15:26:23 UTC, ketmar wrote:

 On Thu, 05 Feb 2015 15:21:08 +0000, Atila Neves wrote:


 the package manager part should have good "machine" interface 
 to allow
 it's usage in various scripts and other build systems. and 
 build
 system... oh, well, it can simply use package manager. this 
 way we can
 discuss "dub-pkg" and "dub-build" as separate entities, and 
 flaws of
 one project will not automatically propagate on another 
 project.


 
 The machine interface is "dub describe". It's JSON, so... 
 that's what my
 Emacs package uses to pass -I flags to dmd for on-the-fly 
 syntax checks.


 there is too much noise there. and there is no easy access to 
 package
 manager itself. as far as i can see, there is no command even 
 to list all
 *known* packages, only installed ones.


Define noise in a M2M context.
Accessing to all known packages wouldn't be hard to implement 
(hint: 
https://github.com/D-Programming-Language/dub/commit/5a93ab440017aa0998c50e
da5cabad7c2bffc5f), 
it just need a dev^W champion.
Feb 05 2015
parent ketmar <ketmar ketmar.no-ip.org> writes:
On Thu, 05 Feb 2015 15:33:51 +0000, Mathias LANG wrote:


 On Thursday, 5 February 2015 at 15:26:23 UTC, ketmar wrote:

 On Thu, 05 Feb 2015 15:21:08 +0000, Atila Neves wrote:


 the package manager part should have good "machine" interface to
 allow it's usage in various scripts and other build systems. and
 build system... oh, well, it can simply use package manager. this way
 we can discuss "dub-pkg" and "dub-build" as separate entities, and
 flaws of one project will not automatically propagate on another
 project.


=20
 The machine interface is "dub describe". It's JSON, so...
 that's what my Emacs package uses to pass -I flags to dmd for
 on-the-fly syntax checks.


 there is too much noise there. and there is no easy access to package
 manager itself. as far as i can see, there is no command even to list
 all *known* packages, only installed ones.


=20
 Define noise in a M2M context.


i don't give a shit about all that 60 lines of crap "dub describe" spits.=20
i'm only interested in "dependencies" parts and... oh, and in compiler=20
flags and pathes, which aren't even there. everything else is just a=20
noise.


 Accessing to all known packages wouldn't be hard to implement (hint:
 https://github.com/D-Programming-Language/dub/


commit/5a93ab440017aa0998c50edda5cabad7c2bffc5f),

 it just need a dev^W champion.


there is no sense in implementing it in dub anyway. at least while dub=20
insists to be a build system.=
Feb 05 2015
prev sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
 there is too much noise there. and there is no easy access to 
 package
 manager itself. as far as i can see, there is no command even 
 to list all
 *known* packages, only installed ones.


You don't need to know all packages, only the ones required by 
the build and where they are. For a build system, anyway, there's 
obviously value in listing all known packages. Adding arbitrary 
dependencies and tasks to the package dependencies is what 
completes the picture.

Atila
Feb 05 2015
parent ketmar <ketmar ketmar.no-ip.org> writes:
On Thu, 05 Feb 2015 15:34:06 +0000, Atila Neves wrote:


 there is too much noise there. and there is no easy access to package
 manager itself. as far as i can see, there is no command even to list
 all *known* packages, only installed ones.


=20
 You don't need to know all packages, only the ones required by the build
 and where they are.


i don't need that, but my "configure" script may need to. ah, isn't list=20
of all *available* packages is a part of the core functionality of=20
package system? ;-)=
Feb 05 2015
prev sibling parent reply =?ISO-8859-15?Q?S=F6nke_Ludwig?= <sludwig rejectedsoftware.com> writes:
Am 05.02.2015 um 16:18 schrieb ketmar:

 On Thu, 05 Feb 2015 16:05:11 +0100, Sönke Ludwig wrote:


 Yes, incremental building is indeed a missing feature. It's simply a
 matter of available developer time, as for many of the other concerns.
 Otherwise this is something that has been acknowledged for inclusion
 basically since the beginning of the project.


 alas, without this feature dub is not a build system. i understand that
 there are alot of things to do and the resources are limited. that's why
 i think that dub should be separated to two projects: package manager and
 build system.

 the package manager part should have good "machine" interface to allow
 it's usage in various scripts and other build systems. and build
 system... oh, well, it can simply use package manager. this way we can
 discuss "dub-pkg" and "dub-build" as separate entities, and flaws of one
 project will not automatically propagate on another project.


That's how it is meant to be! Maybe separating the build functionality 
into a separate executable would indeed be a good idea to express this 
more clearly (even if it wouldn't change anything practically). But the 
main goals were to provide the "dub generate" and "dub describe" 
functionality, so that any build system could be used.

GIT master now supports generating VisualD, CMake and Sublime Text 
project files. "Make" is also a commonly requested one that would be 
easy to add, but hasn't been done yet. Of course it also supports RDMD 
using "dub build --rdmd".
Feb 05 2015
next sibling parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Thu, 2015-02-05 at 16:52 +0100, S=C3=B6nke Ludwig via Digitalmars-d wrot=
e:

=20


[=E2=80=A6]

 GIT master now supports generating VisualD, CMake and Sublime Text=20
 project files. "Make" is also a commonly requested one that would be=20
 easy to add, but hasn't been done yet. Of course it also supports=20
 RDMD using "dub build --rdmd".


What, no SCons.

Also there needs to be a collaboration between Dub developers and=20
Kingsley Hendrickse with his IntelliJ IDEA plugin so as to get IDEA=20
project generation from Dub specs.

Actually probably need to ensure CLion as well as IDEA, but this ought=20
to come for free.

Whatever the Eclipse, Netbeans, IDEA/PyCharm/CLion/Android-Studio=20
history, the current (and very sensible in my opinion) approach is for=20
there to be a separate project and dependency specification, that is=20
then used by the IDE to create a project =E2=80=93 and for there to be=20
updatability. If there is also a non-IDE way of building as well even=20
better.

All (some better than others) support Gradle specifications.=20
Especially true now that the Android toolchain is Gradle/Android-
Studio. Well except for Go folk who find Gradle an abomination.

Dub, to be a player, has to prove better than Gradle for the C/C++/D=20
combination. In a similar position is Go, Rust/Cargo, Ceylon, so it is=20
not all Gradle.

I got a vibe from last Tuesday evening's first London D Users meeting=20
that there is an opportunity for D to compete with the=20
OCAML/Haskell/Python/C++ game in finance industry, well at least the=20
hedge fund and quant end of it.

I think having a really good dependency and build systems is way, way,=20
way more important than improving DMD/GDC/LDC in the next few months.=20
Getting a "this is the new thing about using D" out there is key.


--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Feb 06 2015
prev sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
On Thu, 05 Feb 2015 16:52:53 +0100, S=C3=B6nke Ludwig wrote:


 alas, without this feature dub is not a build system. i understand that
 there are alot of things to do and the resources are limited. that's
 why i think that dub should be separated to two projects: package
 manager and build system.

 the package manager part should have good "machine" interface to allow
 it's usage in various scripts and other build systems. and build
 system... oh, well, it can simply use package manager. this way we can
 discuss "dub-pkg" and "dub-build" as separate entities, and flaws of
 one project will not automatically propagate on another project.


 That's how it is meant to be!


yet this is not clear now. some people looking at dub as "build system=20
with crappy package manager included", other as "package manager with=20
crappy build system included" (and some other as "chimera made of crappy=20
package manager and crappy build system" ;-).

clear separation of two projects will immediately show the goals of each=20
one, so there will be less space for misunderstanding and speculations.=
Feb 06 2015
prev sibling parent reply "Mathias LANG" <geod24 gmail.com> writes:
On Thursday, 5 February 2015 at 15:01:48 UTC, ketmar wrote:

 ??? i didn't modified "mymod.d"! why it is rebuilding it?! 
 'cmon, it's in
 no way better than a simple shell script that just executes 
 "dmd -c" for
 each file and then links the results.


Looks like you missing one of my mails:
http://forum.dlang.org/post/knfueyhpseoviumkewzk forum.dlang.org
Feb 05 2015
parent ketmar <ketmar ketmar.no-ip.org> writes:
On Thu, 05 Feb 2015 15:17:44 +0000, Mathias LANG wrote:


 On Thursday, 5 February 2015 at 15:01:48 UTC, ketmar wrote:

 ??? i didn't modified "mymod.d"! why it is rebuilding it?! 'cmon, it's
 in no way better than a simple shell script that just executes "dmd -c"
 for each file and then links the results.


=20
 Looks like you missing one of my mails:
 http://forum.dlang.org/post/knfueyhpseoviumkewzk forum.dlang.org


yes, i did. yet this is not the only thing dub cannot do. dub is not a=20
build system, dub is a package manager and quickhacked batch builder in=20
one application. and this is the root of all problems: they should be two=20
completely separate things.=
Feb 05 2015
prev sibling parent reply =?ISO-8859-15?Q?S=F6nke_Ludwig?= <sludwig rejectedsoftware.com> writes:
Am 03.02.2015 um 09:51 schrieb ketmar:

 On Tue, 03 Feb 2015 02:19:55 +0000, Martin Nowak wrote:


 There seems to be a general scepticism against dub and I wonder what the
 reasons are.


 'cause it really sux as a build tool.


Just to state the design perspective for a little bit of rationale: DUB 
is not designed to be a build tool itself, but rather to operate on a 
meta level, able to invoke other build tools or to generate foreign 
build descriptions, similar to what CMake and similar systems do. This 
is the reason why its own build system is rather simple (i.e. hasn't 
received enough work to be more comprehensive) and doesn't yet support 
things like incremental builds (there are additional reasons for this, 
though). It's meant to provide a convenient default functionality, but 
doesn't claim to be fully featured.
Feb 05 2015
parent reply "Atila Neves" <atila.neves gmail.com> writes:
On Thursday, 5 February 2015 at 15:46:57 UTC, Sönke Ludwig wrote:

 Am 03.02.2015 um 09:51 schrieb ketmar:

 On Tue, 03 Feb 2015 02:19:55 +0000, Martin Nowak wrote:


 There seems to be a general scepticism against dub and I 
 wonder what the
 reasons are.


 'cause it really sux as a build tool.


 Just to state the design perspective for a little bit of 
 rationale: DUB is not designed to be a build tool itself, but 
 rather to operate on a meta level, able to invoke other build 
 tools or to generate foreign build descriptions, similar to 
 what CMake and similar systems do. This is the reason why its 
 own build system is rather simple (i.e. hasn't received enough 
 work to be more comprehensive) and doesn't yet support things 
 like incremental builds (there are additional reasons for this, 
 though). It's meant to provide a convenient default 
 functionality, but doesn't claim to be fully featured.


Which is fair enough. I want to contribute and write a fully 
featured one, though. Either as a plugin, a self-contained tool 
that builds on dub or something else. How do you suggest I 
proceed, if at all?

Atila
Feb 05 2015
parent =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 05.02.2015 um 16:51 schrieb Atila Neves:

 On Thursday, 5 February 2015 at 15:46:57 UTC, Sönke Ludwig wrote:

 Am 03.02.2015 um 09:51 schrieb ketmar:

 On Tue, 03 Feb 2015 02:19:55 +0000, Martin Nowak wrote:


 There seems to be a general scepticism against dub and I wonder what
 the
 reasons are.


 'cause it really sux as a build tool.


 Just to state the design perspective for a little bit of rationale:
 DUB is not designed to be a build tool itself, but rather to operate
 on a meta level, able to invoke other build tools or to generate
 foreign build descriptions, similar to what CMake and similar systems
 do. This is the reason why its own build system is rather simple (i.e.
 hasn't received enough work to be more comprehensive) and doesn't yet
 support things like incremental builds (there are additional reasons
 for this, though). It's meant to provide a convenient default
 functionality, but doesn't claim to be fully featured.


 Which is fair enough. I want to contribute and write a fully featured
 one, though. Either as a plugin, a self-contained tool that builds on
 dub or something else. How do you suggest I proceed, if at all?

 Atila


I think my personal favorite would be to directly improve the existing 
build system [1]. This would have the strongest impact, as everyone 
using DUB would directly profit. We'd still have to decide if/how the 
build functionality should be separated into another executable or 
shared library, but that wouldn't invalidate the existing work.

Directly creating a separate tool that invokes "dub describe" to get the 
required build information would of course have the advantage of 
explicitly separating building from package meta information gathering 
right from the start. But it has the disadvantage that it requires some 
more work up front to convert the JSON back into a more manageable 
runtime representation and we'd have to think about how this will all 
fit together in the long run (e.g. does the user invoke the build tool 
directly or does DUB do that? Or do we maybe implement some git-like 
plug-in functionality, so that "dub build" calls "dub-build" internally?)

[1]: 
https://github.com/D-Programming-Language/dub/blob/master/source/dub/generators/build.d
Feb 05 2015
prev sibling next sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 2 February 2015 at 08:09:39 UTC, Vladimir Panteleev 
wrote:

 Change my view.


Thanks for the great discussion, everyone.

I agree, Git does not scale for version/dependency management. 
Diamond dependencies are the killer argument. I had not run into 
this because I pile everything reusable in a single repository.

I might revisit Dub again once some of the fixable issues 
mentioned here are fixed.
Feb 02 2015
parent reply "Martin Nowak" <code dawg.eu> writes:
On Tuesday, 3 February 2015 at 02:39:56 UTC, Vladimir Panteleev 
wrote:

 I might revisit Dub again once some of the fixable issues 
 mentioned here are fixed.


Another very important argument is consistency in using packages.
I recently tried digger which is a really great tool, but it took 
me about 5 min to get it running.

- make temp folder
- copy URL from github & clone & cd digger
- search build instructions README/INSTALL/wiki/main.d
- rdmd --build-only digger => linker error
- Wondering, if I really have time for this?
- ah, need to clone all submodules
- rdmd digger --help

With dub you do this instead.

dub fetch digger
dub run digger -- --help

That works in whatever folder you are, builds a stable version of 
the tool and even links against already installed and build 
dependencies.
Feb 02 2015
next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Tuesday, 3 February 2015 at 03:20:49 UTC, Martin Nowak wrote:

 - rdmd --build-only digger => linker error
 - Wondering, if I really have time for this?


Hmm. Fixed ;)

https://github.com/CyberShadow/Digger/commit/b6d06be2346986a74937c918ed1d4ac121a9819f
Feb 02 2015
prev sibling next sibling parent "NVolcz" <niklas.volcz gmail.com> writes:
On Tuesday, 3 February 2015 at 03:20:49 UTC, Martin Nowak wrote:

 On Tuesday, 3 February 2015 at 02:39:56 UTC, Vladimir Panteleev 
 wrote:
 With dub you do this instead.

 dub fetch digger
 dub run digger -- --help


This is the killer feature for me! As a user I don't really care 
about how it is built.
Feb 03 2015
prev sibling parent reply Martin Nowak <code+news.digitalmars dawg.eu> writes:
On 02/03/2015 04:20 AM, Martin Nowak wrote:

 On Tuesday, 3 February 2015 at 02:39:56 UTC, Vladimir Panteleev wrote:

 I might revisit Dub again once some of the fixable issues mentioned
 here are fixed.


 Another very important argument is consistency in using packages.
 I recently tried digger which is a really great tool, but it took me
 about 5 min to get it running.


Just used Digger again and I'm going to copy my console output here, 
because it nicely illustrates my point.

➜ ~ cd Build/Digger/
➜ Digger git:(master) ls
ae/       bisect.ini         cache.d       common.d  custom.d  digger.d 
           digger-web.d  repo/   result/  win32/
bisect.d  bisect.ini.sample  CHANGELOG.md  current/  digger* 
digger.ini.sample  README.md     repo.d  web/
➜ Digger git:(master) git pull
rdremote: Counting objects: 25, done.
remote: Compressing objects: 100% (25/25), done.
remote: Total 25 (delta 11), reused 0 (delta 0)
Unpacking objects: 100% (25/25), done.
 From https://github.com/CyberShadow/Digger
    8f33627..b6d06be  master     -> origin/master
Fetching submodule ae
md --buiFirst, rewinding head to replay your work on top of it...
Fast-forwarded master to b6d06be2346986a74937c918ed1d4ac121a9819f.
➜ Digger git:(master) ✗ rdmd --build-only digger
digger.d(12): Error: module main is in file 'ae/utils/main.d' which 
cannot be read
import path[0] = .
import path[1] = /usr/include/dmd/phobos
import path[2] = /usr/include/dmd/druntime/import
Failed: ["dmd", "-v", "-o-", "digger.d", "-I."]
➜ Digger git:(master) ✗ ls
ae/       bisect.ini         cache.d       common.d  current/  digger* 
  digger.ini.sample  README.md  repo.d   web/
bisect.d  bisect.ini.sample  CHANGELOG.md  config.d  custom.d  digger.d 
  digger-web.d       repo/      result/  win32/
➜ Digger git:(master) ✗ git submodule update
fatal: reference is not a tree: 318ccc76669e9840f43b88e63bcfb1e461ce6d61
Unable to checkout '318ccc76669e9840f43b88e63bcfb1e461ce6d61' in 
submodule path 'ae'
➜ Digger git:(master) ✗ git submodule fetch
usage: git submodule [--quiet] add [-b <branch>] [-f|--force] [--name 
<name>] [--reference <repository>] [--] <repository> [<path>]
    or: git submodule [--quiet] status [--cached] [--recursive] [--] 
[<path>...]
    or: git submodule [--quiet] init [--] [<path>...]
    or: git submodule [--quiet] deinit [-f|--force] [--] <path>...
    or: git submodule [--quiet] update [--init] [--remote] 
[-N|--no-fetch] [-f|--force] [--checkout|--merge|--rebase] [--reference 
<repository>] [--recursive] [--] [<path>...]
    or: git submodule [--quiet] summary [--cached|--files] 
[--summary-limit <n>] [commit] [--] [<path>...]
    or: git submodule [--quiet] foreach [--recursive] <command>
    or: git submodule [--quiet] sync [--recursive] [--] [<path>...]
➜ Digger git:(master) ✗ git submodule status
+c58d5b75242fcab15872acc5d922e103d3ae9423 ae (remotes/upstream/HEAD)
  7a476ca6139869e89943bcf21e0b27904bdf9491 win32 (heads/master)
➜ Digger git:(master) ✗ gitk
➜ Digger git:(master) ✗ git gui
➜ Digger git:(master) ✗ git reset --hard
HEAD is now at b6d06be Detect absence of ae library
➜ Digger git:(master) ✗ git diff
diff --git a/ae b/ae
index 318ccc7..c58d5b7 160000
--- a/ae
+++ b/ae
   -1 +1   
-Subproject commit 318ccc76669e9840f43b88e63bcfb1e461ce6d61
+Subproject commit c58d5b75242fcab15872acc5d922e103d3ae9423
➜ Digger git:(master) ✗ git reset --hard
HEAD is now at b6d06be Detect absence of ae library
➜ Digger git:(master) ✗ rdmd --build-only digger
digger.d(12): Error: module main is in file 'ae/utils/main.d' which 
cannot be read
import path[0] = .
import path[1] = /usr/include/dmd/phobos
import path[2] = /usr/include/dmd/druntime/import
Failed: ["dmd", "-v", "-o-", "digger.d", "-I."]
➜ Digger git:(master) ✗ cd ae
➜ ae gitk
➜ ae cd ..
➜ Digger git:(master) ✗ git submodule sync
Synchronizing submodule url for 'ae'
Synchronizing submodule url for 'win32'
➜ Digger git:(master) ✗ cat .gitmodules
[submodule "ae"]
	path = ae
	url = git://github.com/CyberShadow/ae.git
[submodule "win32"]
	path = win32
	url = https://github.com/CS-svnmirror/dsource-bindings-win32
➜ Digger git:(master) ✗ cat .git
.git/  .gitignore  .gitmodules
➜ Digger git:(master) ✗ rdmd --build-only digger
digger.d(12): Error: module main is in file 'ae/utils/main.d' which 
cannot be read
import path[0] = .
import path[1] = /usr/include/dmd/phobos
import path[2] = /usr/include/dmd/druntime/import
Failed: ["dmd", "-v", "-o-", "digger.d", "-I."]
➜ Digger git:(master) ✗ git submodule update
remote: Counting objects: 8, done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 8 (delta 5), reused 4 (delta 1)
Unpacking objects: 100% (8/8), done.
 From git://github.com/CyberShadow/ae
    217b080..66502df  master     -> origin/master
    e8d867c..0b1cfc0  new-net    -> origin/new-net
  * [new branch]      old-net    -> origin/old-net
Submodule path 'ae': checked out '318ccc76669e9840f43b88e63bcfb1e461ce6d61'
➜ Digger git:(master) rdmd --build-only digger
➜ Digger git:(master)

No idea why git submodule update worked the second time.
Feb 03 2015
next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Wednesday, 4 February 2015 at 01:04:42 UTC, Martin Nowak wrote:

 On 02/03/2015 04:20 AM, Martin Nowak wrote:

 On Tuesday, 3 February 2015 at 02:39:56 UTC, Vladimir 
 Panteleev wrote:

 I might revisit Dub again once some of the fixable issues 
 mentioned
 here are fixed.


 Another very important argument is consistency in using 
 packages.




Some people are more familiar with git, some with Dub. As a Dub 
developer, I'm not surprised you're more familiar with it. The 
two simple commands you quoted look as alien to me as the 
supplied build instructions would look to someone who never used 
git - meanwhile, they are self-explanatory to everyone who's used 
git submodules before.

I'm sure I or anyone could at will produce similar logs of 
failing to figure out dub basics and it spitting mysterious error 
messages.

Frankly, I don't see this as a considerable argument.
Feb 03 2015
prev sibling parent "Mathias LANG" <geod24 gmail.com> writes:
On Wednesday, 4 February 2015 at 01:04:42 UTC, Martin Nowak wrote:

 No idea why git submodule update worked the second time.


You can simply use: `git submodule update --init`, which init 
submodules if they aren't yet.
Better, write yourself an alias for clone, let's call it 'cl', 
which includes --recursive. I used to have one that cloned from 
github at a specific tag (git gcl Geod24 phobos 2.067 would clone 
phobos in Geod24-phobos-2.067). I wasn't aware this would break 
some code.

submodules can be nice to work with, but you have to get use to 
it. Which is why I love dub, it lowers the entry barrier.
Howerver, provided the feedback on this thread, it could use a 
step-by-step tutorial.
Feb 04 2015
prev sibling next sibling parent reply "Martin Nowak" <code dawg.eu> writes:
Lot's of weird attitude in this thread BTW, instead of 
complaining about every single flaw we could really make use of a 
few helping hands.
I'm often literally baffled that people don't even try to fix 
obviously trivial bugs.
https://github.com/D-Programming-Language/dub/pull/354
Feb 02 2015
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Tue, 03 Feb 2015 02:44:01 +0000, Martin Nowak wrote:


 I'm often literally baffled that people don't even try to fix obviously
 trivial bugs.


maybe that's 'cause they have their work to do, and fixing bugs in dub is=20
not a part of that work?=
Feb 03 2015
parent reply "ponce" <contact gam3sfrommars.fr> writes:
On Tuesday, 3 February 2015 at 08:54:04 UTC, ketmar wrote:

 On Tue, 03 Feb 2015 02:44:01 +0000, Martin Nowak wrote:


 I'm often literally baffled that people don't even try to fix 
 obviously
 trivial bugs.


 maybe that's 'cause they have their work to do, and fixing bugs 
 in dub is
 not a part of that work?


Never enough time to fix bugs, always enough time to complain on 
the newsgroupd.
Feb 03 2015
parent ketmar <ketmar ketmar.no-ip.org> writes:
On Tue, 03 Feb 2015 10:26:13 +0000, ponce wrote:


 On Tuesday, 3 February 2015 at 08:54:04 UTC, ketmar wrote:

 On Tue, 03 Feb 2015 02:44:01 +0000, Martin Nowak wrote:


 I'm often literally baffled that people don't even try to fix
 obviously trivial bugs.


 maybe that's 'cause they have their work to do, and fixing bugs in dub
 is not a part of that work?


=20
 Never enough time to fix bugs, always enough time to complain on the
 newsgroupd.


sure. fixing bug means reading and understanding the code. making patch.=20
testing that patch. alot of time and distraction. and NG posting is easy.=
Feb 03 2015
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-02-02 09:09, Vladimir Panteleev wrote:


 1a. rdmd and D's module system:

 When you run `dmd -o- program.d`, the compiler will automatically read
 all modules imported by your program, and their imports, and so on. It
 does so by searching the filesystem across its search path for matches
 which correspond with D's module system, and only reads those files that
 are needed.


This doesn't work for libraries. For example, two files that don't 
import each other.

-- 
/Jacob Carlborg
Feb 04 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Wednesday, 4 February 2015 at 18:03:54 UTC, Jacob Carlborg 
wrote:

 On 2015-02-02 09:09, Vladimir Panteleev wrote:


 1a. rdmd and D's module system:

 When you run `dmd -o- program.d`, the compiler will 
 automatically read
 all modules imported by your program, and their imports, and 
 so on. It
 does so by searching the filesystem across its search path for 
 matches
 which correspond with D's module system, and only reads those 
 files that
 are needed.


 This doesn't work for libraries. For example, two files that 
 don't import each other.


You mean via extern(C)?

You can use pragma(lib), and the compiler will emit a linker 
instruction to add the specified library. I'm not sure about the 
platform support, though.
Feb 04 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-02-04 19:07, Vladimir Panteleev wrote:


 You mean via extern(C)?

 You can use pragma(lib), and the compiler will emit a linker instruction
 to add the specified library. I'm not sure about the platform support,
 though.


No, I mean like this:

module mylib.foo;

void foo () {}


module mylib.bar;

void bar () {}


$ rdmd mylib/foo.d

The above command will obviously not compile "bar.d". What I mean is 
that it's no longer enough to pass a single file and let RDMD track the 
dependencies.

-- 
/Jacob Carlborg
Feb 08 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 8 February 2015 at 21:02:14 UTC, Jacob Carlborg wrote:

 The above command will obviously not compile "bar.d". What I 
 mean is that it's no longer enough to pass a single file and 
 let RDMD track the dependencies.


OK, but the obviously trivial fix is to either import bar, or 
create a module that imports all other modules in the library. 
It's not really enough justification for switching build tools, 
is it?
Feb 08 2015
next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 9 February 2015 at 07:15:23 UTC, Vladimir Panteleev 
wrote:

 On Sunday, 8 February 2015 at 21:02:14 UTC, Jacob Carlborg 
 wrote:

 The above command will obviously not compile "bar.d". What I 
 mean is that it's no longer enough to pass a single file and 
 let RDMD track the dependencies.


 OK, but the obviously trivial fix is to either import bar, or 
 create a module that imports all other modules in the library. 
 It's not really enough justification for switching build tools, 
 is it?


Is it really any different justification from avoiding ./source 
folder? :)
Feb 09 2015
parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 9 February 2015 at 08:08:36 UTC, Dicebot wrote:

 On Monday, 9 February 2015 at 07:15:23 UTC, Vladimir Panteleev 
 wrote:

 On Sunday, 8 February 2015 at 21:02:14 UTC, Jacob Carlborg 
 wrote:

 The above command will obviously not compile "bar.d". What I 
 mean is that it's no longer enough to pass a single file and 
 let RDMD track the dependencies.


 OK, but the obviously trivial fix is to either import bar, or 
 create a module that imports all other modules in the library. 
 It's not really enough justification for switching build 
 tools, is it?


 Is it really any different justification from avoiding ./source 
 folder? :)


Yes. As I said dozens of times already, using a source folder is 
currently incompatible with git submodules (unless you use 
add-local). Using a module list is not incompatible with anything.
Feb 09 2015
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-02-09 08:15, Vladimir Panteleev wrote:


 OK, but the obviously trivial fix is to either import bar, or create a
 module that imports all other modules in the library. It's not really
 enough justification for switching build tools, is it?


I thought the whole point of this thread was that you don't like to 
change you're code structure/workflow, i.e. one should not adapt the 
code to the tool. Now you're suggesting I do that?

-- 
/Jacob Carlborg
Feb 09 2015
parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 9 February 2015 at 08:10:56 UTC, Jacob Carlborg wrote:

 On 2015-02-09 08:15, Vladimir Panteleev wrote:


 OK, but the obviously trivial fix is to either import bar, or 
 create a
 module that imports all other modules in the library. It's not 
 really
 enough justification for switching build tools, is it?


 I thought the whole point of this thread was that you don't 
 like to change you're code structure/workflow, i.e. one should 
 not adapt the code to the tool. Now you're suggesting I do that?


No, it's not. Care to read it again?

By your logic, a dub.json file or a Makefile is not code, but a 
.d file which contains nothing but import statements is suddenly 
code and you're changing your workflow to adapt to rdmd? Please.
Feb 09 2015
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-02-09 08:15, Vladimir Panteleev wrote:


 OK, but the obviously trivial fix is to either import bar, or create a
 module that imports all other modules in the library. It's not really
 enough justification for switching build tools, is it?


Setting up dependencies between modules just to satisfy a build tool 
doesn't sound like the right solution. In Phobos we're currently trying 
to minimize the dependencies between modules

-- 
/Jacob Carlborg
Feb 09 2015
parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 9 February 2015 at 08:16:24 UTC, Jacob Carlborg wrote:

 On 2015-02-09 08:15, Vladimir Panteleev wrote:


 OK, but the obviously trivial fix is to either import bar, or 
 create a
 module that imports all other modules in the library. It's not 
 really
 enough justification for switching build tools, is it?


 Setting up dependencies between modules just to satisfy a build 
 tool doesn't sound like the right solution. In Phobos we're 
 currently trying to minimize the dependencies between modules


Ugh... what does that have to do with anything? How is a 
meta-module which imports every other module in a library 
different from a Makefile or a dub.json file which explicitly 
lists all .d files in the project - other than have a .d 
extension?
Feb 09 2015
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-02-02 09:09, Vladimir Panteleev wrote:


 Even if I had faith that dub was a perfectly polished piece of software,
 it doesn't solve any problems I have with building D programs, and in
 fact would make said task more complicated. Here's why.

 1. rdmd

 rdmd is a simple and effective way to build D programs, and I'm sad to
 see its use declining.

 rdmd leverages two things: D's module system, and the compiler's import
 search path. Dub's design seems to ignore both of the above.


I think one of the biggest advantage of Dub is the registry, code.dlang.org.

Another thing is it supports a cross-platform way of configure a build. 
Just take a simple thing as linking a static library will most likely 
look different on different platforms. Also, you most likely need a 
wrapper script that calls rdmd with all arguments. There's basically no 
language that works on both Windows and Posix out of the box. The only 
choice is to either go with one file for Posix (shell script) and one 
for Windows (batch files). Or you could go with D, which seems a bit 
overkill for just a passing a couple of flags.

-- 
/Jacob Carlborg
Feb 04 2015
next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Wednesday, 4 February 2015 at 19:15:16 UTC, Jacob Carlborg 
wrote:

 Another thing is it supports a cross-platform way of configure 
 a build. Just take a simple thing as linking a static library 
 will most likely look different on different platforms. Also, 
 you most likely need a wrapper script that calls rdmd with all 
 arguments. There's basically no language that works on both 
 Windows and Posix out of the box. The only choice is to either 
 go with one file for Posix (shell script) and one for Windows 
 (batch files). Or you could go with D, which seems a bit 
 overkill for just a passing a couple of flags.


I've been using a build script for RABCDAsm since 2010, it works 
quite well. But then, I don't think it would be possible to 
replace it with a dub.json file.

https://github.com/CyberShadow/RABCDAsm/blob/master/build_rabcdasm.d
Feb 04 2015
prev sibling next sibling parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Wed, 2015-02-04 at 20:15 +0100, Jacob Carlborg via Digitalmars-d wrote:

 


[…]

 I think one of the biggest advantage of Dub is the registry, 
 code.dlang.org.


The success of Maven was not Maven the build system, it was Maven 
Central the artefact repository. OK, now superceded by JCenter, but…
-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Feb 04 2015
prev sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
On Wednesday, 4 February 2015 at 19:15:16 UTC, Jacob Carlborg 
wrote:

 On 2015-02-02 09:09, Vladimir Panteleev wrote:


 Even if I had faith that dub was a perfectly polished piece of 
 software,
 it doesn't solve any problems I have with building D programs, 
 and in
 fact would make said task more complicated. Here's why.

 1. rdmd

 rdmd is a simple and effective way to build D programs, and 
 I'm sad to
 see its use declining.

 rdmd leverages two things: D's module system, and the 
 compiler's import
 search path. Dub's design seems to ignore both of the above.


 I think one of the biggest advantage of Dub is the registry, 
 code.dlang.org.

 Another thing is it supports a cross-platform way of configure 
 a build. Just take a simple thing as linking a static library 
 will most likely look different on different platforms. Also, 
 you most likely need a wrapper script that calls rdmd with all 
 arguments. There's basically no language that works on both 
 Windows and Posix out of the box. The only choice is to either 
 go with one file for Posix (shell script) and one for Windows 
 (batch files). Or you could go with D, which seems a bit 
 overkill for just a passing a couple of flags.


It depends on how you define "out of the box", I guess. I can 
(and have) easily build C and C++ on Windows, Linux and Mac OS X 
with CMake. Static library? Here you go:

add_library(mylib STATIC foo.cpp bar.cpp)

Done. Pretty sure it's the same with premake, gradle, ...

D might be overkill for passing a couple of flags, but then you 
always have rdmd. Unless you have package dependencies, then you 
use dub, with the already mentioned caveats. But it's not 
overkill when your build has an autogenerated file that gets 
built from a D binary that includes some other files which...

You get the idea. dub's lack of features is only mildly annoying 
for my current D projects, they're small. I end up waiting a 
little longer for builds. But if I had a large codebase, which I 
would if I had a D-based startup, I'd need a hell of a lot more.

Atila
Feb 05 2015
parent Jacob Carlborg <doob me.com> writes:
On 2015-02-05 09:56, Atila Neves wrote:


 It depends on how you define "out of the box", I guess.


I define it as what's included in a newly installed operating system. 
The biggest issue here is Windows vs Posix.

-- 
/Jacob Carlborg
Feb 08 2015
prev sibling next sibling parent reply "weaselcat" <weaselcat gmail.com> writes:
After reading this thread I think I'm the only person here who 
actually likes makefiles.

Nothing ever feels as complete as a good old hand written 
makefile.
Feb 04 2015
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/4/15 10:46 PM, weaselcat wrote:

 After reading this thread I think I'm the only person here who actually
 likes makefiles.

 Nothing ever feels as complete as a good old hand written makefile.


I'm don't mind makefiles. That says, our dmd/druntime/phobos/dlang.org 
makefiles break at the drop of a hat. It's also rather difficult to 
review changes to them because the implications are not always obvious. 
-- Andrei
Feb 04 2015
parent reply "Atila Neves" <atila.neves gmail.com> writes:
On Thursday, 5 February 2015 at 07:13:44 UTC, Andrei Alexandrescu 
wrote:

 On 2/4/15 10:46 PM, weaselcat wrote:

 After reading this thread I think I'm the only person here who 
 actually
 likes makefiles.

 Nothing ever feels as complete as a good old hand written 
 makefile.


 I'm don't mind makefiles. That says, our 
 dmd/druntime/phobos/dlang.org makefiles break at the drop of a 
 hat. It's also rather difficult to review changes to them 
 because the implications are not always obvious. -- Andrei


Would a transition to something more maintainable be welcomed?

Atila
Feb 05 2015
parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Atila Neves"  wrote in message news:wpryuzyjmcdjtocqgcnw forum.dlang.org...


 I'm don't mind makefiles. That says, our dmd/druntime/phobos/dlang.org 
 makefiles break at the drop of a hat. It's also rather difficult to 
 review changes to them because the implications are not always 
 obvious. -- Andrei


 Would a transition to something more maintainable be welcomed?


A transition to a D-based script would.  Adding a dependency on another 
build tool less so.
Feb 05 2015
parent reply "Atila Neves" <atila.neves gmail.com> writes:
On Thursday, 5 February 2015 at 09:46:44 UTC, Daniel Murphy wrote:

 "Atila Neves"  wrote in message 
 news:wpryuzyjmcdjtocqgcnw forum.dlang.org...


 I'm don't mind makefiles. That says, our 
 dmd/druntime/phobos/dlang.org makefiles break at the drop of 
 a hat. It's also rather difficult to review changes to them 
 because the implications are not always obvious. -- Andrei


 Would a transition to something more maintainable be welcomed?


 A transition to a D-based script would.  Adding a dependency on 
 another build tool less so.


I understand why dependencies are to be avoided. The good thing 
about the state of things right now is that one needs only a C++ 
compiler and make. That's a good thing.

Atila
Feb 05 2015
parent "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Atila Neves"  wrote in message news:myzbnoipafejrpzdwgnf forum.dlang.org...


 I understand why dependencies are to be avoided. The good thing about the 
 state of things right now is that one needs only a C++ compiler and make. 
 That's a good thing.


It certainly is.  We are however planning to require a D compiler as well.
Feb 05 2015
prev sibling next sibling parent reply "Paulo Pinto" <pjmlp progtools.org> writes:
On Thursday, 5 February 2015 at 06:46:12 UTC, weaselcat wrote:

 After reading this thread I think I'm the only person here who 
 actually likes makefiles.

 Nothing ever feels as complete as a good old hand written 
 makefile.


As long as you just need one compiler, one OS, one CPU 
architecture and no other resources. And are the only developer 
touching them.

Otherwise the path to portable and maintanble Makefiles is a road 
full with many perils.
Feb 04 2015
parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 5 February 2015 at 07:20, Paulo Pinto via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Thursday, 5 February 2015 at 06:46:12 UTC, weaselcat wrote:

 After reading this thread I think I'm the only person here who actually
 likes makefiles.

 Nothing ever feels as complete as a good old hand written makefile.



 As long as you just need one compiler, one OS, one CPU architecture and no
 other resources. And are the only developer touching them.

 Otherwise the path to portable and maintanble Makefiles is a road full with
 many perils.


I've recently had fun with configure scripts.

Questions such as: "Why are we looking for libiberty?" (it's not used
anywhere); "Why are we doing C++ compiler tests?" (only gdc and gcc
are used to build phobos/druntime); "Why are we testing time.h and
other system headers for various platform features?" (It probably
won't do much good failing the configure step, as this information
will likely be out of step with druntime's C bindings).

Every once in a while its worth doing a sweep - you may end up 1600
lines of shell scripting better off than before. :o)
Feb 05 2015
parent ketmar <ketmar ketmar.no-ip.org> writes:
On Thu, 05 Feb 2015 08:26:14 +0000, Iain Buclaw via Digitalmars-d wrote:


 I've recently had fun with configure scripts.


ah, crapfest in all it's glory. the whole autocrap thing was made by=20
martians, for the sake of torturing humankind.=
Feb 05 2015
prev sibling next sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
On Thursday, 5 February 2015 at 06:46:12 UTC, weaselcat wrote:

 After reading this thread I think I'm the only person here who 
 actually likes makefiles.

 Nothing ever feels as complete as a good old hand written 
 makefile.


Assuming you get it right. In my experience, this is analogous to 
saying "nothing ever feels as complete as manually managing 
memory". Humans shouldn't write Makefiles.

As in the case of managing memory, I used to do that (hand-write 
Makefiles). I don't anymore because the alternatives are far 
better.

Atila
Feb 05 2015
parent "deadalnix" <deadalnix gmail.com> writes:
On Thursday, 5 February 2015 at 09:00:05 UTC, Atila Neves wrote:

 On Thursday, 5 February 2015 at 06:46:12 UTC, weaselcat wrote:

 After reading this thread I think I'm the only person here who 
 actually likes makefiles.

 Nothing ever feels as complete as a good old hand written 
 makefile.


 Assuming you get it right. In my experience, this is analogous 
 to saying "nothing ever feels as complete as manually managing 
 memory". Humans shouldn't write Makefiles.

 As in the case of managing memory, I used to do that 
 (hand-write Makefiles). I don't anymore because the 
 alternatives are far better.

 Atila


And just as manual memory management, human must be able to 
manage it manually when required. Not in the general case, not as 
the default, but as a possibility.

This is what is wrong with most other build systems.
Feb 06 2015
prev sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
On Thu, 05 Feb 2015 06:46:10 +0000, weaselcat wrote:


 After reading this thread I think I'm the only person here who actually
 likes makefiles.
=20
 Nothing ever feels as complete as a good old hand written makefile.


i bet you love writing your programs in assembler...=
Feb 05 2015
prev sibling next sibling parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 02.02.2015 um 09:09 schrieb Vladimir Panteleev:

 Even if I had faith that dub was a perfectly polished piece of software,
 it doesn't solve any problems I have with building D programs, and in
 fact would make said task more complicated. Here's why.

 (...)


I think I've already answered most of the points in other replies, so 
I'm just going to skip to the final list of claims.


 So, in order to start using Dub, I'd need to:

 - restructure the directory layout of my library (breaking change)


Not true. Use "sourcePaths" and "importPaths" to configure the package 
as appropriate.


 - update all projects which use this library to use Dub instead


Not true. Adding a dub.json will probably not break any dependent projects.


 - give up quick syntax checking


Not true. Use path based dependencies or "add-path"/"add-local" to 
reference dependencies wherever you want.


 - give up commit-granularity versioning


True. However, this can be fixed. Direct GIT/HG support is already on 
the list of nice to have features.


 - begin maintaining JSON configuration files


Well, for most projects that would mean writing a tiny JSON file once 
and never touch it again. If you commit to it, you can of course stop 
maintaining a D build script, so I'm not sure how much of an argument 
this is.


 - begin versioning libraries by hand


A good thing, IMHO! Commit hashes alone have no semantical meaning, 
which is problematic for public libraries for many reasons. For private 
libraries things can be different and additionally supporting commit 
based dependencies may be good to have.


 - install Dub on all my computers, servers, and virtual machines


Yes. You could then skip installing GIT, though ;)


 No thanks.

 I could invest time in improving Dub to fix or ameliorate some of the
 above points, but I don't see a compelling reason to. In fact, I think
 we should integrate rdmd into dmd - dmd clearly already knows which
 source files participate in compilation, as all rdmd does is basically
 take dmd's output and feed it back to it. This will greatly speed up
 compilation, too.

 Change my view.


DUB in it's current form focuses on three primary points:

  - Building a package should "just work", including building the 
dependencies (mind the bugs)
  - Provide a public package registry with minimal effort to register 
and maintain packages
  - Minimize the barrier to entry for "usual" projects

The more advanced use cases, such as interfacing with other languages, 
will be the focus of future development. The initial primary goals above 
have been chosen to be most helpful for language newcomers, so that we 
get the maximum gain for the language ecosystem as a whole (in terms of 
growth and barrier to entry). More advanced users and users with legacy 
code bases are of course just as important in the long run, so those 
concerns need to be addressed now, too (as developer resources permit).

Regarding a DMD integrated version of RDMD, I always have thought the 
same. If that were the case, using a recursive build strategy would 
definitely be the mode of operation for DUB, too. The reason why it 
works differently today is simply that it has to support all the build 
systems/project file formats of the supported IDEs/build systems and 
most of them work in a classic C++-like way with a list of source files.

IDE support was also one of the primary goals, because there are large 
groups of developers that exclusively develop with IDEs and forcing them 
to the command line would simply make it much more likely that they 
won't use D at all. On top of that I personally very much like the 
ability to quickly build with Visual Studio to use its graphical debugger.
Feb 05 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Thursday, 5 February 2015 at 14:48:24 UTC, Sönke Ludwig wrote:

 - restructure the directory layout of my library (breaking 
 change)


 Not true. Use "sourcePaths" and "importPaths" to configure the 
 package as appropriate.


Right now this is necessary to maintain directory/package path 
consistency.


 - update all projects which use this library to use Dub instead


 Not true. Adding a dub.json will probably not break any 
 dependent projects.


The problem is certainly not with the addition of dub.json, but 
with the change in directory layout.


 - give up quick syntax checking


 Not true. Use path based dependencies or "add-path"/"add-local" 
 to reference dependencies wherever you want.


Ah, good idea.


 - begin maintaining JSON configuration files


 Well, for most projects that would mean writing a tiny JSON 
 file once and never touch it again. If you commit to it, you 
 can of course stop maintaining a D build script, so I'm not 
 sure how much of an argument this is.


I think the only build script I had to write was for RABCDAsm, 
which probably can't be reimplemented as a dub.json file:

http://forum.dlang.org/post/peurhbfebwdskcballzk forum.dlang.org

For DFeed and a few small projects I use Makefiles and packaging 
scripts, but they also do more complicated things such as signing 
Windows executables, compressing/compiling resources, and 
applying .manifest files.

In general, I've tried to make things work with rdmd alone.


 - begin versioning libraries by hand


 A good thing, IMHO! Commit hashes alone have no semantical 
 meaning, which is problematic for public libraries for many 
 reasons. For private libraries things can be different and 
 additionally supporting commit based dependencies may be good 
 to have.


I can see how that is more practical for public libraries. Commit 
granularity helps if you want reproducible builds, though.
Feb 05 2015
parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 05.02.2015 um 16:15 schrieb Vladimir Panteleev:

 On Thursday, 5 February 2015 at 14:48:24 UTC, Sönke Ludwig wrote:

 - restructure the directory layout of my library (breaking change)


 Not true. Use "sourcePaths" and "importPaths" to configure the package
 as appropriate.


 Right now this is necessary to maintain directory/package path consistency.


(Still reading the thread) If you mean that fetched packages are stored 
within "<packagename>-<packageversion>" folders, that's true of course, 
I didn't think of that. So it wouldn't work for a publicly registered 
package, but it would still work if you just use the library as a git 
submodule, or using some other kind of checkout protocol that keeps the 
original folder name. I think we should look into a fix here, such as 
using "<packagename>-<packageversion>/<packagename>".


 - begin maintaining JSON configuration files


 Well, for most projects that would mean writing a tiny JSON file once
 and never touch it again. If you commit to it, you can of course stop
 maintaining a D build script, so I'm not sure how much of an argument
 this is.


 I think the only build script I had to write was for RABCDAsm, which
 probably can't be reimplemented as a dub.json file:

 http://forum.dlang.org/post/peurhbfebwdskcballzk forum.dlang.org

 For DFeed and a few small projects I use Makefiles and packaging
 scripts, but they also do more complicated things such as signing
 Windows executables, compressing/compiling resources, and applying
 .manifest files.

 In general, I've tried to make things work with rdmd alone.


Okay, things like testing for DMD bugs and signing stuff would indeed 
still have to be done outside of DUB using a separate 
pre(Build/Generate)Command (a failing command should automatically stop 
the build), but at least the dependency management part (haveLZMA) 
should map fine to DUB packages.

I agree that if you only look at it in this context, that DUB may look 
like a useless complication (two build scripts/files instead of one). 
The real advantage comes when things like complex dependency trees or 
IDE project files get involved.


 - begin versioning libraries by hand


 A good thing, IMHO! Commit hashes alone have no semantical meaning,
 which is problematic for public libraries for many reasons. For
 private libraries things can be different and additionally supporting
 commit based dependencies may be good to have.


 I can see how that is more practical for public libraries. Commit
 granularity helps if you want reproducible builds, though.


Fun fact: If you register a GIT working copy using "dub 
add-path/add-local" and it doesn't have an exact tag checked out, then 
any package that uses it, will have something like this in its 
dub.selections.json file: '"vibe-d": "0.7.22+commit.17.g75ecb6b"', where 
"0.7.22" is the latest version tag on that branch and the suffix is the 
current commit hash.

This information is currently not used to drive GIT and automatically 
check out that same commit before building. But we could of course 
implement that in one form or another to have fully reproducible builds 
with commit granularity.
Feb 05 2015
parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Thursday, 5 February 2015 at 16:32:26 UTC, Sönke Ludwig wrote:

 I think the only build script I had to write was for RABCDAsm, 
 which
 probably can't be reimplemented as a dub.json file:

 http://forum.dlang.org/post/peurhbfebwdskcballzk forum.dlang.org


 Okay, things like testing for DMD bugs and signing stuff would 
 indeed still have to be done outside of DUB using a separate 
 pre(Build/Generate)Command (a failing command should 
 automatically stop the build), but at least the dependency 
 management part (haveLZMA) should map fine to DUB packages.


The haveLZMA thing isn't needed to detect D packages, but the 
presence of the C library, similar to what a ./configure script 
does.


 Fun fact: If you register a GIT working copy using "dub 
 add-path/add-local" and it doesn't have an exact tag checked 
 out, then any package that uses it, will have something like 
 this in its dub.selections.json file: '"vibe-d": 
 "0.7.22+commit.17.g75ecb6b"', where "0.7.22" is the latest 
 version tag on that branch and the suffix is the current commit 
 hash.

 This information is currently not used to drive GIT and 
 automatically check out that same commit before building. But 
 we could of course implement that in one form or another to 
 have fully reproducible builds with commit granularity.


That's nice. Having the build system generate a file with the 
dependencies' commits should work as well as git submodules, if 
it can be expected that the build system will be invoked at least 
once before committing.
Feb 05 2015
prev sibling next sibling parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 02.02.2015 um 09:09 schrieb Vladimir Panteleev:

 On Monday, 2 February 2015 at 05:23:52 UTC, Daniel Murphy wrote:
 1. rdmd


BTW, there is one thing about RDMD that can be a real issue and one that 
is not easily solved without integrating its functionality directly into 
DMD: It doesn't track local and string imports. This is the main reason 
why I haven't personally used it since a while, although it is directly 
supported with "dub --rdmd".
Feb 05 2015
next sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
On Thursday, 5 February 2015 at 15:01:57 UTC, Sönke Ludwig wrote:

 Am 02.02.2015 um 09:09 schrieb Vladimir Panteleev:

 On Monday, 2 February 2015 at 05:23:52 UTC, Daniel Murphy 
 wrote:
 1. rdmd


 BTW, there is one thing about RDMD that can be a real issue and 
 one that is not easily solved without integrating its 
 functionality directly into DMD: It doesn't track local and 
 string imports. This is the main reason why I haven't 
 personally used it since a while, although it is directly 
 supported with "dub --rdmd".


String imports: just checked. It does. It uses dmd to get the 
imports so I thought it'd be weird if it didn't.

What's a local import? You mean this?

void func() {
     import mymodule;
}

Because I just tried that and it worked, too.

Atila
Feb 05 2015
parent =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 05.02.2015 um 16:12 schrieb Atila Neves:

 On Thursday, 5 February 2015 at 15:01:57 UTC, Sönke Ludwig wrote:

 Am 02.02.2015 um 09:09 schrieb Vladimir Panteleev:

 On Monday, 2 February 2015 at 05:23:52 UTC, Daniel Murphy wrote:
 1. rdmd


 BTW, there is one thing about RDMD that can be a real issue and one
 that is not easily solved without integrating its functionality
 directly into DMD: It doesn't track local and string imports. This is
 the main reason why I haven't personally used it since a while,
 although it is directly supported with "dub --rdmd".


 String imports: just checked. It does. It uses dmd to get the imports so
 I thought it'd be weird if it didn't.

 What's a local import? You mean this?

 void func() {
      import mymodule;
 }

 Because I just tried that and it worked, too.

 Atila


Exactly, see also Vladimir's ticket: 
http://d.puremagic.com/issues/show_bug.cgi?id=7016

It only seems to happen for imported modules and not for the one passed 
directly to DMD, AFAICS.
Feb 05 2015
prev sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Thursday, 5 February 2015 at 15:01:57 UTC, Sönke Ludwig wrote:

 BTW, there is one thing about RDMD that can be a real issue and 
 one that is not easily solved without integrating its 
 functionality directly into DMD: It doesn't track local and 
 string imports. This is the main reason why I haven't 
 personally used it since a while, although it is directly 
 supported with "dub --rdmd".


String imports were fixed recently, I think.

Local imports:

http://d.puremagic.com/issues/show_bug.cgi?id=7016
Feb 05 2015
parent =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 05.02.2015 um 16:16 schrieb Vladimir Panteleev:

 On Thursday, 5 February 2015 at 15:01:57 UTC, Sönke Ludwig wrote:

 BTW, there is one thing about RDMD that can be a real issue and one
 that is not easily solved without integrating its functionality
 directly into DMD: It doesn't track local and string imports. This is
 the main reason why I haven't personally used it since a while,
 although it is directly supported with "dub --rdmd".


 String imports were fixed recently, I think.


Good to know. This would make it practical to use for me again.


 Local imports:

 http://d.puremagic.com/issues/show_bug.cgi?id=7016
Feb 05 2015
prev sibling parent Martin Nowak <code+news.digitalmars dawg.eu> writes:
On 02/02/2015 09:09 AM, Vladimir Panteleev wrote:

 Change my view.


Most important point for dub, there is central registry that enables the 
http://en.wikipedia.org/wiki/Network_effect.
Feb 13 2015