www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - How do people feel about putting source compiler directives inside

reply aliak <something something.com> writes:
Like these in rund? 
https://github.com/dragon-lang/rund#source-compiler-directives

Is that something that dlang would be open to for rdmd?

Cheers
- Ali
Dec 02 2019
next sibling parent reply bachmeier <no spam.net> writes:
On Monday, 2 December 2019 at 18:48:41 UTC, aliak wrote:
 Like these in rund? 
 https://github.com/dragon-lang/rund#source-compiler-directives

 Is that something that dlang would be open to for rdmd?

 Cheers
 - Ali
What is the advantage relative to the added complexity? Can't you use Dub or a Makefile if that's what you want? I'm disagreeing, I just don't know the use case.
Dec 02 2019
parent reply aliak <something something.com> writes:
On Monday, 2 December 2019 at 19:32:58 UTC, bachmeier wrote:
 On Monday, 2 December 2019 at 18:48:41 UTC, aliak wrote:
 Like these in rund? 
 https://github.com/dragon-lang/rund#source-compiler-directives

 Is that something that dlang would be open to for rdmd?

 Cheers
 - Ali
What is the advantage relative to the added complexity? Can't you use Dub or a Makefile if that's what you want? I'm disagreeing, I just don't know the use case.
Dub is painfully slow for scripting, and if you require a makefile whenever someone wants to write a script then python it is. The usecase is being able to have your scripts use the compiler flags you want (e.g. dip1000 and optimize, or whatever else).
Dec 02 2019
parent reply bachmeier <no spam.net> writes:
On Monday, 2 December 2019 at 20:04:17 UTC, aliak wrote:

 Dub is painfully slow for scripting
Dub is painful for everything else so I'm not surprised to hear it's painful for scripting.
 if you require a makefile whenever someone wants to write a 
 script then python it is.
If that's all it takes for someone to use Python, they'll be using Python regardless. I can understand the idea of rdmd if you can use it without configuration. If you're going to add compiler configuration information to your "script", you no longer have a script, you have something really confusing for Python users.
Dec 02 2019
next sibling parent reply mipri <mipri minimaltype.com> writes:
On Monday, 2 December 2019 at 21:43:17 UTC, bachmeier wrote:
 On Monday, 2 December 2019 at 20:04:17 UTC, aliak wrote:

 Dub is painfully slow for scripting
Dub is painful for everything else so I'm not surprised to hear it's painful for scripting.
I have a bash script that uses 'dub run' to invoke dfmt, and the chief problem with that is that any random use of dfmt may trigger a slow library upgrade. If shebang uses of dub work like this as well, that's a big problem for scripting. In the more limited use of dub scripts that I've used, they're fine. Caching is very similar to rdmd. I'm not fond of /tmp as a default (it's often noexec) but there's no variation here in the tooling.
 if you require a makefile whenever someone wants to write a
 script then python it is.
If that's all it takes for someone to use Python, they'll be using Python regardless. I can understand the idea of rdmd if you can use it without configuration. If you're going to add compiler configuration information to your "script", you no longer have a script, you have something really confusing for Python users.
A script is a file with source code in it that you can execute. The essential feature of scripts is that the source code is what you're executing. In deployed systems, on servers, in other people's code, when you come across a binary that you have a problem with, it can be a long adventure to find the source of that binary, and then you may still have not found *the* source to *the* binary, but just *some* source that could create similar binaries. For example, if you find a repo link in a near by document, what commit to that repo was used to build the binary that you have? With a script, the source is always there. This helps enormously in troubleshooting, especially hands-on kind of troubleshooting where you temporary modify the program to see how this affects what you're investigating. If someone has coded in an ill-advised restriction, you can temporarily bypass it. If there's a bug and the fix can't be deployed for whatever reason, you can tell people about a sed oneliner that fixes the script in situ on machines where they need the fix. You may not even have access to see the repo that the script's source is in, but since you have the script you can fix a problem and email the author a patch. The documentation is also often always there, as it can be inline with the script. You may be able to 'perldoc /root/bin/special_sysadmin_thing' and get more detailed notes about the exact version of the exact thing at hand. Compiler configuration flags change nothing at all about this.
Dec 02 2019
next sibling parent reply bachmeier <no spam.net> writes:
On Monday, 2 December 2019 at 22:18:06 UTC, mipri wrote:

 A script is a file with source code in it that you can execute.
 The essential feature of scripts is that the source code is what
 you're executing.
If you include compiler directives inside the file, you're *not* executing the code, because you're adding code to tell the compiler what to do with the file. You lose portability once you add compilation information.
 In deployed systems, on servers, in other people's code, when
 you come across a binary that you have a problem with, it can be
 a long adventure to find the source of that binary, and then you
 may still have not found *the* source to *the* binary, but just
 *some* source that could create similar binaries. For example,
 if you find a repo link in a near by document, what commit to
 that repo was used to build the binary that you have?
You can add a comment explaining how to compile the code. The proposed change to rdmd would require a bunch of added complexity, and probably introduce bugs, all in the name of duplicating Makefile functionality. Let's leave currently working tools alone.
Dec 02 2019
next sibling parent reply mipri <mipri minimaltype.com> writes:
On Tuesday, 3 December 2019 at 01:28:00 UTC, bachmeier wrote:
 On Monday, 2 December 2019 at 22:18:06 UTC, mipri wrote:

 A script is a file with source code in it that you can execute.
 The essential feature of scripts is that the source code is
 what
 you're executing.
If you include compiler directives inside the file, you're *not* executing the code, because you're adding code to tell the compiler what to do with the file.
There's no operational difference. The user types ./foo, stuff happens. If the user edits foo and then types ./foo again, something different happens. The illusion break for D is that some invocations are slower than others due to compilation overhead. Python had (or has) the same effect, though less noticeable, with its .pyc cache files. This is bearable and compiler directives do not change it in any way.
 You lose portability
 once you add compilation information.
This is actually an offensively ridiculous concern.
 In deployed systems, on servers, in other people's code, when
 you come across a binary that you have a problem with, it can
 be
 a long adventure to find the source of that binary, and then
 you
 may still have not found *the* source to *the* binary, but just
 *some* source that could create similar binaries. For example,
 if you find a repo link in a near by document, what commit to
 that repo was used to build the binary that you have?
You can add a comment explaining how to compile the code.
Comments are not code, and when you compile a script you get a binary, not a script. The exact same problems can arise: I have a problem with this binary and the source is nowhere to be found. Or I have a problem with this binary and the nearby script has a comment that may or may have not been followed to build it.
 The
 proposed change to rdmd would require a bunch of added
 complexity,
Adding code adds complexity. This is true.
 probably introduce bugs
Adding code probably adds bugs. This is also true.
 duplicating Makefile functionality.
Makefiles have nothing to do with scripts.
 Let's leave currently
 working tools alone.
rdmd is not a "working tool" when your task requires some compiler flags. This level of scare-mongering about some trivial array manipulation is absurd to the point that I actually start to suspect malicious intentions. Who sent you and how much are they paying you? Suppose a General asked for input on how to assault a position, and a Colonel responded by waxing about the very serious hazards posed by the very small rocks to be found in the desert near the position? "Why, if a soldier were to kneel down during the battle to inspect one of these rocks, he might scratch a finger. It is also conceivable that some arrangement of small rocks might make for slippery footing. For these reasons we should take the much more heavily defended western approach, which--" Are you serious? Are you actually serious?
Dec 02 2019
next sibling parent mipri <mipri minimaltype.com> writes:
On Tuesday, 3 December 2019 at 02:02:08 UTC, mipri wrote:
 This level of scare-mongering about some trivial array
 manipulation is absurd to the point that I actually start to
 suspect malicious intentions. Who sent you and how much are
 they paying you?
I wasn't kidding about this, but a more charitable interpretation is: you don't use scripts, don't much care about rdmd, and just want resources to go towards something else instead. "c'mon guys, have you heard of makefiles? This is a solved problem. Please work on something instead of this rdmd thing that I'm not sure anyone even uses." Due to the restricted communication channel (text) and certain internet habits (everyone's spouting off like an authority about everything, so weakly-stated claims are easily lost in the noise) this reasonable wish can come across as I took it. Well, not to worry: 1. This feature would only affect people using scripts, and realistically any bugs would also only affect people using the feature. 2. Technically it's easy enough that pretty much anyone could do it. It'd be a good task for me, for example, when I probably can't contribute to whatever you're thinking about as a more desirable development. So more relevant objections are stuff like "wouldn't letting someone add -J in a script let them steal /etc/passwd with a string import?" (there's no reason to give the dmd binary setuid privs or anything so compile-time and run-time access should be the same, but maybe there's some crazy circumstance where this matters with selinux or OpenBSD stuff).
Dec 02 2019
prev sibling parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Tuesday, 3 December 2019 at 02:02:08 UTC, mipri wrote:
 On Tuesday, 3 December 2019 at 01:28:00 UTC, bachmeier wrote:
 On Monday, 2 December 2019 at 22:18:06 UTC, mipri wrote:

 A script is a file with source code in it that you can 
 execute.
 The essential feature of scripts is that the source code is
 what
 you're executing.
If you include compiler directives inside the file, you're *not* executing the code, because you're adding code to tell the compiler what to do with the file.
There's no operational difference. The user types ./foo, stuff happens. If the user edits foo and then types ./foo again, something different happens. The illusion break for D is that some invocations are slower than others due to compilation overhead. Python had (or has) the same effect, though less noticeable, with its .pyc cache files. This is bearable and compiler directives do not change it in any way.
 You lose portability
 once you add compilation information.
This is actually an offensively ridiculous concern.
 In deployed systems, on servers, in other people's code, when
 you come across a binary that you have a problem with, it can
 be
 a long adventure to find the source of that binary, and then
 you
 may still have not found *the* source to *the* binary, but 
 just
 *some* source that could create similar binaries. For example,
 if you find a repo link in a near by document, what commit to
 that repo was used to build the binary that you have?
You can add a comment explaining how to compile the code.
Comments are not code, and when you compile a script you get a binary, not a script. The exact same problems can arise: I have a problem with this binary and the source is nowhere to be found. Or I have a problem with this binary and the nearby script has a comment that may or may have not been followed to build it.
 The
 proposed change to rdmd would require a bunch of added
 complexity,
Adding code adds complexity. This is true.
 probably introduce bugs
Adding code probably adds bugs. This is also true.
 duplicating Makefile functionality.
Makefiles have nothing to do with scripts.
 Let's leave currently
 working tools alone.
rdmd is not a "working tool" when your task requires some compiler flags. This level of scare-mongering about some trivial array manipulation is absurd to the point that I actually start to suspect malicious intentions. Who sent you and how much are they paying you?
I've had the same thoughts and feelings many times on this forum and in the Dlang community in general. I've learned to spend less time trying to convince people so I can spend more time doing what I love, which is programming. If I can't convince someone, or they are being unreasonable then I figure out how I can get what I want without their permission. It usually means I must spend more time learning and possibly duplicating effort, but in my experience the trade-off is usually worth it. After so much time arguing with people about rdmd, I created rund. As a result, I now have a much faster and better tool that I've been using for over a year, and I've been spared a large amount of frustration. Since I sometimes use this tool hundreds of times a day it was worth the investment, and I'm very happy with the result. If others want to use it, improve it, or add it to package management systems then I'm happy to work with them and will be glad to see it help others as well. P.S. you mentioned the 'noexec' issue, I noticed this a while back and opened an issue on it for rund (https://github.com/dragon-lang/rund/issues/17). I have a few solutions but no one's really asked for it yet so I haven't chosen a solution yet. When someone brings up a need and/or use case for it then I'll be implementing one of the solutions.
Dec 02 2019
parent Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Tuesday, 3 December 2019 at 06:55:41 UTC, Jonathan Marler 
wrote:
 I've had the same thoughts and feelings many times on this 
 forum and in the Dlang community in general.  I've learned to 
 spend less time trying to convince people so I can spend more 
 time doing what I love, which is programming.  If I can't 
 convince someone, or they are being unreasonable then I figure 
 out how I can get what I want without their permission.  It 
 usually means I must spend more time learning and possibly 
 duplicating effort, but in my experience the trade-off is 
 usually worth it.
I understand where you're coming from. I really appreciate the amount and quality of work that you have contributed to D over the years. I think it's quite unfortunate if D's community can't reach a consensus and work together, instead of being divided (or even just duplicating effort).
 After so much time arguing with people about rdmd, I created 
 rund.  As a result, I now have a much faster and better tool 
 that I've been using for over a year, and I've been spared a 
 large amount of frustration.  Since I sometimes use this tool 
 hundreds of times a day it was worth the investment, and I'm 
 very happy with the result.  If others want to use it, improve 
 it, or add it to package management systems then I'm happy to 
 work with them and will be glad to see it help others as well.
Given the amount of good work invested in D build tools, such as rdmd, rund, build.d and dub, it's really sad that they're becoming a source of contention and divisive among the D community. I believe that the best thing that can happen to D is if all the effort that's invested across all of them gets merged into a single code base, maintained by all authors and endorsed by the community as whole. I see no reason why we couldn't work on a shared code base that it's as easy and fast to use as rund, offer larger-scale development features, such as dependency management and also offer direct and expressive way to manually declare new targets, like build.d.
Dec 03 2019
prev sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Tuesday, 3 December 2019 at 01:28:00 UTC, bachmeier wrote:
 On Monday, 2 December 2019 at 22:18:06 UTC, mipri wrote:

 A script is a file with source code in it that you can execute.
 The essential feature of scripts is that the source code is 
 what
 you're executing.
If you include compiler directives inside the file, you're *not* executing the code, because you're adding code to tell the compiler what to do with the file. You lose portability once you add compilation information.
On the contrary. Without compiler directives or a tool that has them built-in, like dub or rdmd/rund you don't have any portability, because you have no idea how run build the script. It's ridiculous to expect just the right set dependencies to be magically installed on the system where the script will run. Instead good scripts should contain all the necessary information to be run in a self-sufficient and cross-platform manner. This can include the compiler version (e.g. DMD 2.085.0 or LDC 1.19.0-beta2), the build tool (e.g. dub 1.16.0), the build flags (e.g. -dip1000) and any libraries that they depend on (e.g. phobos 2.089.0, mir-algorithm 3.7.2).
 In deployed systems, on servers, in other people's code, when
 you come across a binary that you have a problem with, it can 
 be
 a long adventure to find the source of that binary, and then 
 you
 may still have not found *the* source to *the* binary, but just
 *some* source that could create similar binaries. For example,
 if you find a repo link in a near by document, what commit to
 that repo was used to build the binary that you have?
You can add a comment explaining how to compile the code.
A mechanically verifiable and executable directive is infinitely better ;)
 The proposed change to rdmd would require a bunch of added 
 complexity, and probably introduce bugs, all in the name of 
 duplicating Makefile functionality. Let's leave currently 
 working tools alone.
IMO these use cases already solved by dub and rund, so I think there's no need to duplicate the functionality. The functionality is very much worth it there.
Dec 03 2019
parent reply bachmeier <no spam.net> writes:
On Tuesday, 3 December 2019 at 09:46:59 UTC, Petar Kirov 
[ZombineDev] wrote:
 On the contrary. Without compiler directives or a tool that has 
 them built-in, like dub or rdmd/rund you don't have any 
 portability, because you have no idea how run build the script. 
 It's ridiculous to expect just the right set dependencies to be 
 magically installed on the system where the script will run.
None of this is relevant if you have a tool that is limited to running code that doesn't need compiler directives. I'm aware that some people think you have to add every feature to every tool. I disagree. Especially in cases where the functionality is already provided by other tools.
Dec 03 2019
next sibling parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Tuesday, 3 December 2019 at 14:01:07 UTC, bachmeier wrote:
 On Tuesday, 3 December 2019 at 09:46:59 UTC, Petar Kirov 
 [ZombineDev] wrote:
 On the contrary. Without compiler directives or a tool that 
 has them built-in, like dub or rdmd/rund you don't have any 
 portability, because you have no idea how run build the 
 script. It's ridiculous to expect just the right set 
 dependencies to be magically installed on the system where the 
 script will run.
None of this is relevant if you have a tool that is limited to running code that doesn't need compiler directives. I'm aware that some people think you have to add every feature to every tool. I disagree. Especially in cases where the functionality is already provided by other tools.
This is true in most cases. The problem is that some requirements of a D program can't be configured in the source file. They can only be specified on the command-line. For example, if you have a D program that requires -betterC, you couldn't declare this source file itself. You can't specify import paths, versions and many other options in the source file. Not all programs need to specify any of these options, but some do. Here's an example of a D script that requires import paths, a version, betterC, and no dmd.conf file: https://github.com/dragon-lang/mar/blob/master/test/go.d Without "Source Compiler Directives" this source file would not be able to declare the options that are required to build/run it correctly. These requirements would need to be satisfied outside the source file itself, either through user intervention on the command-line or through some other tool. Some D scripts have requirements that can only be specified via compiler flags. I would rather have a way to specify these in the language but for now this feature solves the issue without having to make any big changes to the language.
Dec 03 2019
parent reply bachmeier <no spam.net> writes:
On Tuesday, 3 December 2019 at 16:45:52 UTC, Jonathan Marler 
wrote:

 None of this is relevant if you have a tool that is limited to 
 running code that doesn't need compiler directives. I'm aware 
 that some people think you have to add every feature to every 
 tool. I disagree. Especially in cases where the functionality 
 is already provided by other tools.
This is true in most cases. The problem is that some requirements of a D program can't be configured in the source file. They can only be specified on the command-line. For example, if you have a D program that requires -betterC, you couldn't declare this source file itself. You can't specify import paths, versions and many other options in the source file.
Why is it necessary to do this with rdmd? Aren't there already tools that do this? I'm honestly surprised that someone using -betterC would even want a script.
 Not all programs need to specify any of these options, but some 
 do.  Here's an example of a D script that requires import 
 paths, a version, betterC, and no dmd.conf file:

 https://github.com/dragon-lang/mar/blob/master/test/go.d
But you were able to do that without rdmd.
Dec 03 2019
parent reply aliak <something something.com> writes:
On Tuesday, 3 December 2019 at 18:47:06 UTC, bachmeier wrote:
 On Tuesday, 3 December 2019 at 16:45:52 UTC, Jonathan Marler 
 wrote:

 None of this is relevant if you have a tool that is limited 
 to running code that doesn't need compiler directives. I'm 
 aware that some people think you have to add every feature to 
 every tool. I disagree. Especially in cases where the 
 functionality is already provided by other tools.
This is true in most cases. The problem is that some requirements of a D program can't be configured in the source file. They can only be specified on the command-line. For example, if you have a D program that requires -betterC, you couldn't declare this source file itself. You can't specify import paths, versions and many other options in the source file.
Why is it necessary to do this with rdmd? Aren't there already tools that do this? I'm honestly surprised that someone using -betterC would even want a script.
There are always going to be people who use tools in ways that you don't - probably, the majority of use cases are not your use cases. While person A will make a makefile and source file, and have the user download it, run make, install, and then be able to run it, person B will use a tool that allows their users to "just run it".
 Not all programs need to specify any of these options, but 
 some do.  Here's an example of a D script that requires import 
 paths, a version, betterC, and no dmd.conf file:

 https://github.com/dragon-lang/mar/blob/master/test/go.d
But you were able to do that without rdmd.
You asked for compiler source directive use cases. And yes, point being he can't use rdmd for writing the scripts he needs to write.
Dec 03 2019
parent reply bachmeier <no spam.net> writes:
On Tuesday, 3 December 2019 at 20:54:19 UTC, aliak wrote:

 You asked for compiler source directive use cases.

 And yes, point being he can't use rdmd for writing the scripts 
 he needs to write.
I don't find it convincing to see use cases that are already done with other tools. Making big changes to a tool that's worked well for years in order to duplicate existing functionality isn't IMO a good strategy. The good news for you is that you don't need to convince me. I'm not the one that makes these decisions. Whether I agree or disagree is of no consequence.
Dec 03 2019
parent reply wjoe <invalid example.com> writes:
On Tuesday, 3 December 2019 at 18:39:12 UTC, bachmeier wrote:
 [...] rdmd works, so let's not break it without a really good 
 reason, and I haven't seen a really good reason.
No, it doesn't. The script isn't executed, or if it is there's no effect, but the exit code doesn't indicate an error either. if /tmp is mounted noexec. On Wednesday, 4 December 2019 at 00:40:42 UTC, bachmeier wrote:
 On Tuesday, 3 December 2019 at 20:54:19 UTC, aliak wrote:

 You asked for compiler source directive use cases.

 And yes, point being he can't use rdmd for writing the scripts 
 he needs to write.
I don't find it convincing to see use cases that are already done with other tools.
So you're saying that ldc and gdc are duplicated and hence wasted effort ? In my mind having options and alternatives is as good a thing as diversity can be.
Dec 09 2019
parent bachmeier <no spam.net> writes:
On Monday, 9 December 2019 at 12:19:31 UTC, wjoe wrote:

 I don't find it convincing to see use cases that are already 
 done with other tools.
So you're saying that ldc and gdc are duplicated and hence wasted effort ? In my mind having options and alternatives is as good a thing as diversity can be.
ldc and gdc aren't really good examples, because those projects are separate from dmd. Their existence doesn't break dmd. "More options" is a wonderful idea if you can make those changes *and* someone will fix the bugs and handle additional maintenance issues that arise as a result of the changes that were made. It's not a wonderful idea based on the historical record of this community. We'll end up with something that's sort of usable and gives new users yet another reason to stay away. But as I already said, I'm not the one you have to convince. I'm just a user of the language that prefers unbroken tools.
Dec 09 2019
prev sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Tuesday, 3 December 2019 at 14:01:07 UTC, bachmeier wrote:
 On Tuesday, 3 December 2019 at 09:46:59 UTC, Petar Kirov 
 [ZombineDev] wrote:
 On the contrary. Without compiler directives or a tool that 
 has them built-in, like dub or rdmd/rund you don't have any 
 portability, because you have no idea how run build the 
 script. It's ridiculous to expect just the right set 
 dependencies to be magically installed on the system where the 
 script will run.
None of this is relevant if you have a tool that is limited to running code that doesn't need compiler directives. I'm aware that some people think you have to add every feature to every tool. I disagree. Especially in cases where the functionality is already provided by other tools.
I don't follow, are you arguing that worse is better? Why would one use an inferior tool, if there's another one providing a strict superset of features (or at least a superset of the ones that one needs) at the same level or better performance? Unless one is in the unlikely position of writing a single source file (with no imports/includes) bare-metal application I don't see a situation where one doesn't have any dependencies. Even shell scripts have implicit dependencies on things like the shell interpreter (dash vs bash vs zsh vs fish vs ...), its version, the available executable files in $PATH, other environment variables, and other things. Instead, I believe every program meant to be used by more than one person should contain complete dependency information, so building and running it is completely reproducible. For example, I was one of the supporters of a Dub feature [0] using which you specify the requirements on the D toolchain like this: toolchainRequirements dub=">=1.14.0" frontend=">=2.080 <2.090" This one of reason why I use NixOS [1][2][3]. [0]: https://dlang.org/changelog/2.085.0.html#toolchain_requirements [1]: https://nixos.org/ [2]: https://nixos.org/nix/ [3]: https://nixos.wiki/wiki/NixOS
Dec 03 2019
parent bachmeier <no spam.net> writes:
On Tuesday, 3 December 2019 at 17:16:07 UTC, Petar Kirov 
[ZombineDev] wrote:
 On Tuesday, 3 December 2019 at 14:01:07 UTC, bachmeier wrote:
 On Tuesday, 3 December 2019 at 09:46:59 UTC, Petar Kirov 
 [ZombineDev] wrote:
 On the contrary. Without compiler directives or a tool that 
 has them built-in, like dub or rdmd/rund you don't have any 
 portability, because you have no idea how run build the 
 script. It's ridiculous to expect just the right set 
 dependencies to be magically installed on the system where 
 the script will run.
None of this is relevant if you have a tool that is limited to running code that doesn't need compiler directives. I'm aware that some people think you have to add every feature to every tool. I disagree. Especially in cases where the functionality is already provided by other tools.
I don't follow, are you arguing that worse is better?
I'm saying we have different definitions of "worse". In your opinion, the only thing that matters is the number of features, and that's fine. In my opinion, we need to account for the ease of use of a tool, particularly for beginners, the difficulty of maitenance of the tool, the possibility of clashing with current and future features, and the number of bugs users encounter. rdmd works, so let's not break it without a really good reason, and I haven't seen a really good reason.
 Why would one use an inferior tool, if there's another one 
 providing a strict superset of features (or at least a superset 
 of the ones that one needs) at the same level or better 
 performance?
Nobody would use an inferior tool (that's an interesting choice of language) but I don't agree fewer features implies it's inferior.
 Unless one is in the unlikely position of writing a single 
 source file (with no imports/includes) bare-metal application I 
 don't see a situation where one doesn't have any dependencies. 
 Even shell scripts have implicit dependencies on things like 
 the shell interpreter (dash vs bash vs zsh vs fish vs ...), its 
 version, the available executable files in $PATH, other 
 environment variables, and other things.
rdmd has worked well for many years without the proposed duplication of features.
Dec 03 2019
prev sibling parent Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Monday, 2 December 2019 at 22:18:06 UTC, mipri wrote:
 I have a bash script that uses 'dub run' to invoke dfmt, and the
 chief problem with that is that any random use of dfmt may
 trigger a slow library upgrade. If shebang uses of dub work like
 this as well, that's a big problem for scripting. In the more
 limited use of dub scripts that I've used, they're fine. Caching
 is very similar to rdmd.
The first time `dub run dfmt` is run, dfmt will be fetched and built. After that you can run it like this: dub run dfmt --skip-registry=standard -- <DFMT args> to skip network checks.
 I'm not fond of /tmp as a default (it's often noexec) but
 there's no variation here in the tooling.
Dub keeps packages and build artifacts in `$HOME/.dub/packages`, and in my experience this works really well.
 if you require a makefile whenever someone wants to write a
 script then python it is.
If that's all it takes for someone to use Python, they'll be using Python regardless. I can understand the idea of rdmd if you can use it without configuration. If you're going to add compiler configuration information to your "script", you no longer have a script, you have something really confusing for Python users.
A script is a file with source code in it that you can execute. The essential feature of scripts is that the source code is what you're executing. In deployed systems, on servers, in other people's code, when you come across a binary that you have a problem with, it can be a long adventure to find the source of that binary, and then you may still have not found *the* source to *the* binary, but just *some* source that could create similar binaries. For example, if you find a repo link in a near by document, what commit to that repo was used to build the binary that you have? With a script, the source is always there. This helps enormously in troubleshooting, especially hands-on kind of troubleshooting where you temporary modify the program to see how this affects what you're investigating. If someone has coded in an ill-advised restriction, you can temporarily bypass it. If there's a bug and the fix can't be deployed for whatever reason, you can tell people about a sed oneliner that fixes the script in situ on machines where they need the fix. You may not even have access to see the repo that the script's source is in, but since you have the script you can fix a problem and email the author a patch. The documentation is also often always there, as it can be inline with the script. You may be able to 'perldoc /root/bin/special_sysadmin_thing' and get more detailed notes about the exact version of the exact thing at hand. Compiler configuration flags change nothing at all about this.
I completely agree.
Dec 03 2019
prev sibling next sibling parent aliak <something something.com> writes:
On Monday, 2 December 2019 at 21:43:17 UTC, bachmeier wrote:
 On Monday, 2 December 2019 at 20:04:17 UTC, aliak wrote:

 Dub is painfully slow for scripting
Dub is painful for everything else so I'm not surprised to hear it's painful for scripting.
 if you require a makefile whenever someone wants to write a 
 script then python it is.
If that's all it takes for someone to use Python, they'll be using Python regardless. I can understand the idea of rdmd if you can use it without configuration. If you're going to add compiler configuration information to your "script", you no longer have a script, you have something really confusing for Python users.
This is not true. I'd rather use D for scripting, but not if it requires the user to deal with make files - that's just compiling then. Your other point is also untrue, node had a 'use strict' interpreter directive for a long time, which people used willingly because it made the script code better. For the script user there was no difference (they just use the script, they don't care if it uses dip1000 or not).
Dec 02 2019
prev sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Monday, 2 December 2019 at 21:43:17 UTC, bachmeier wrote:
 On Monday, 2 December 2019 at 20:04:17 UTC, aliak wrote:

 Dub is painfully slow for scripting
On the contrary, I think it's great for what it offers. I just made a quick test and it contributed to to 0.3sec slow down, compared to `dmd -run` of minimal D script (0.21sec vs 0.51sec). I think that could be improved, but still it's a small price to given its feature set. There's no alternative tool for D that empowers single-file scripts to use third-party libraries with zero fiddling with import paths, dependency downloading, build order, linking and so on.
 Dub is painful for everything else so I'm not surprised to hear 
 it's painful for scripting.
Dub is probably the best thing that happened to D in the past 5 years. Yes, it has its share of problems, but I don't think any of them are unfixable or by design. Instead of bashing it, I think this discussion could be more productive if you shared your criticism in a constrictive way.
Dec 03 2019
next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 3 December 2019 at 09:10:16 UTC, Petar Kirov 
[ZombineDev] wrote:
 [snip]
 Dub is probably the best thing that happened to D in the past 5 
 years. [snip]
I agree. Dub is not perfect, but it is a great thing.
Dec 03 2019
prev sibling next sibling parent reply aliak <something something.com> writes:
On Tuesday, 3 December 2019 at 09:10:16 UTC, Petar Kirov 
[ZombineDev] wrote:
 On Monday, 2 December 2019 at 21:43:17 UTC, bachmeier wrote:
 On Monday, 2 December 2019 at 20:04:17 UTC, aliak wrote:

 Dub is painfully slow for scripting
On the contrary, I think it's great for what it offers. I just made a quick test and it contributed to to 0.3sec slow down, compared to `dmd -run` of minimal D script (0.21sec vs 0.51sec). I think that could be improved, but still it's a small price to given its feature set. There's no alternative tool for D that empowers single-file scripts to use third-party libraries with zero fiddling with import paths, dependency downloading, build order, linking and so on.
The script I run takes over a second each invocation (when doing nothing). With rdmd that happens once and then it uses the same binary (and is instant). Do you know if dub caches the binary? If it does then it's just dependency checking. And if that can be turned off then I believe the experience can be satisfactory. But right now it's painful.
Dec 03 2019
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Tuesday, 3 December 2019 at 13:07:04 UTC, aliak wrote:
 On Tuesday, 3 December 2019 at 09:10:16 UTC, Petar Kirov 
 [ZombineDev] wrote:
 On Monday, 2 December 2019 at 21:43:17 UTC, bachmeier wrote:
 On Monday, 2 December 2019 at 20:04:17 UTC, aliak wrote:

 Dub is painfully slow for scripting
On the contrary, I think it's great for what it offers. I just made a quick test and it contributed to to 0.3sec slow down, compared to `dmd -run` of minimal D script (0.21sec vs 0.51sec). I think that could be improved, but still it's a small price to given its feature set. There's no alternative tool for D that empowers single-file scripts to use third-party libraries with zero fiddling with import paths, dependency downloading, build order, linking and so on.
The script I run takes over a second each invocation (when doing nothing). With rdmd that happens once and then it uses the same binary (and is instant). Do you know if dub caches the binary? If it does then it's just dependency checking. And if that can be turned off then I believe the experience can be satisfactory. But right now it's painful.
Dub is supposed to cache every build artifact, but for some reason the `isUpToDate` function returns `false` for a single-file package I am testing with. BTW, on what OS and hardware are you testing? I am on Linux with an SSD, so this may explain why I get faster results than you. Dub has everything necessary to reach execution time parity up +/- 5% with rdmd. The dependency checking is relatively fast for me, but if necessary that can be easily disabled, using the `--skip-registry` option. If it's not already like that IMO dub shouldn't check for updates for single-file packages. For projects with dub.selections.json AFAIK it doesn't check for updates if dub.selection.json contains the versions of transitive dependencies and they had already been fetched. I'll look into the problem with Dub's `isUpToDate`.
Dec 03 2019
next sibling parent aliak <something something.com> writes:
On Tuesday, 3 December 2019 at 16:33:32 UTC, Petar Kirov 
[ZombineDev] wrote:
 Dub is supposed to cache every build artifact, but for some 
 reason the `isUpToDate` function returns `false` for a 
 single-file package I am testing with.

 BTW, on what OS and hardware are you testing?
 I am on Linux with an SSD, so this may explain why I get faster 
 results than you.
I'm on this year's macbook pro with catalina (latest macOS), ssd as well.
 Dub has everything necessary to reach execution time parity up 
 +/- 5% with rdmd.
 The dependency checking is relatively fast for me, but if 
 necessary that can be easily disabled, using the 
 `--skip-registry` option.
 If it's not already like that IMO dub shouldn't check for 
 updates for single-file packages. For projects with 
 dub.selections.json AFAIK it doesn't check for updates if 
 dub.selection.json contains the versions of transitive 
 dependencies and they had already been fetched.

 I'll look into the problem with Dub's `isUpToDate`.
Cool! I think if that's all it is then even a 1 sec initial run is not horrible.
Dec 03 2019
prev sibling next sibling parent reply Andre Pany <andre s-e-a-p.de> writes:
On Tuesday, 3 December 2019 at 16:33:32 UTC, Petar Kirov 
[ZombineDev] wrote:
 On Tuesday, 3 December 2019 at 13:07:04 UTC, aliak wrote:
 [...]
Dub is supposed to cache every build artifact, but for some reason the `isUpToDate` function returns `false` for a single-file package I am testing with. BTW, on what OS and hardware are you testing? I am on Linux with an SSD, so this may explain why I get faster results than you. Dub has everything necessary to reach execution time parity up +/- 5% with rdmd. The dependency checking is relatively fast for me, but if necessary that can be easily disabled, using the `--skip-registry` option. If it's not already like that IMO dub shouldn't check for updates for single-file packages. For projects with dub.selections.json AFAIK it doesn't check for updates if dub.selection.json contains the versions of transitive dependencies and they had already been fetched. I'll look into the problem with Dub's `isUpToDate`.
I testet (dub master) the isUpToDate behavior on windows, linux (shebang and no shebang), with source code files having *.d extension and without. isUpToDate is always working fine and uses the last build. Kind regards André
Dec 03 2019
parent reply Andre Pany <andre s-e-a-p.de> writes:
On Wednesday, 4 December 2019 at 06:12:07 UTC, Andre Pany wrote:
 On Tuesday, 3 December 2019 at 16:33:32 UTC, Petar Kirov 
 [ZombineDev] wrote:
 [...]
I testet (dub master) the isUpToDate behavior on windows, linux (shebang and no shebang), with source code files having *.d extension and without. isUpToDate is always working fine and uses the last build. Kind regards André
This causing the issue: https://github.com/dlang/dub/issues/1809
Dec 03 2019
parent reply Andre Pany <andre s-e-a-p.de> writes:
On Wednesday, 4 December 2019 at 06:39:27 UTC, Andre Pany wrote:
 On Wednesday, 4 December 2019 at 06:12:07 UTC, Andre Pany wrote:
 On Tuesday, 3 December 2019 at 16:33:32 UTC, Petar Kirov 
 [ZombineDev] wrote:
 [...]
I testet (dub master) the isUpToDate behavior on windows, linux (shebang and no shebang), with source code files having *.d extension and without. isUpToDate is always working fine and uses the last build. Kind regards André
This causing the issue: https://github.com/dlang/dub/issues/1809
Here the fix which avoids the rebuild of Single file packages https://github.com/dlang/dub/pull/1811 Dub is a very good piece of software and issues can in most cases easily be fixed. Kind regards Andre
Dec 05 2019
parent reply mipri <mipri minimaltype.com> writes:
On Friday, 6 December 2019 at 05:50:38 UTC, Andre Pany wrote:
 Here the fix which avoids the rebuild of Single file packages 
 https://github.com/dlang/dub/pull/1811
Nice work! running a script with a single dependency, average of 3 runs. currently: Time : 4224 ms (1408.000 ms/per) Max RSS : 313.6 MB and with your pull request: Time : 398 ms (132.667 ms/per) Max RSS : 29.0 MB
 Dub is a very good piece of software and issues can in most 
 cases easily be fixed.

 Kind regards
 Andre
Dec 05 2019
parent reply mipri <mipri minimaltype.com> writes:
On Friday, 6 December 2019 at 06:14:38 UTC, mipri wrote:
 On Friday, 6 December 2019 at 05:50:38 UTC, Andre Pany wrote:
 Here the fix which avoids the rebuild of Single file packages 
 https://github.com/dlang/dub/pull/1811
Nice work! running a script with a single dependency, average of 3 runs. currently: Time : 4224 ms (1408.000 ms/per) Max RSS : 313.6 MB and with your pull request: Time : 398 ms (132.667 ms/per) Max RSS : 29.0 MB
Although... Time : 24 ms (8.000 ms/per) Max RSS : 5.0 MB This is the time I get from directly calling the binary that dub leaves under /tmp So that's still quite a lot of overhead.
 Dub is a very good piece of software and issues can in most 
 cases easily be fixed.

 Kind regards
 Andre
Dec 05 2019
parent reply Andre Pany <andre s-e-a-p.de> writes:
On Friday, 6 December 2019 at 06:45:58 UTC, mipri wrote:
 On Friday, 6 December 2019 at 06:14:38 UTC, mipri wrote:
 On Friday, 6 December 2019 at 05:50:38 UTC, Andre Pany wrote:
 Here the fix which avoids the rebuild of Single file packages 
 https://github.com/dlang/dub/pull/1811
Nice work! running a script with a single dependency, average of 3 runs. currently: Time : 4224 ms (1408.000 ms/per) Max RSS : 313.6 MB and with your pull request: Time : 398 ms (132.667 ms/per) Max RSS : 29.0 MB
Although... Time : 24 ms (8.000 ms/per) Max RSS : 5.0 MB This is the time I get from directly calling the binary that dub leaves under /tmp So that's still quite a lot of overhead.
 Dub is a very good piece of software and issues can in most 
 cases easily be fixed.

 Kind regards
 Andre
What is the overhead of dub, if you do not count the first run? (the first run compiles the script to an executable, each sub subsequent run from now on just calls the executable) Did you compile dub using dmd or ldc? Performance of LDC with optimizations on is a lot better than debug builds of dmd. Kind regards André
Dec 05 2019
parent reply mipri <mipri minimaltype.com> writes:
On Friday, 6 December 2019 at 07:47:33 UTC, Andre Pany wrote:
 What is the overhead of dub, if you do not count the first run?
 (the first run compiles the script to an executable, each sub 
 subsequent
 run from now on just calls the executable)
It's calling dmd on every run, even with your patch: 1575620540.004204 execve("/usr/bin/dmd", ["dmd", "-quiet", "-c", "-o-", "-v", "long /tmp filename.d"] 1575620540.204598 execve("long tmp filename" ... but that seems to be probePlatform, which stopWatch says only takes 12ms. Statting the generated file shows that it's not getting touched, so it's definitely not getting recompiled.
 Did you compile dub using dmd or ldc? Performance of LDC with 
 optimizations
 on is a lot better than debug builds of dmd.
dmd, but a release build with ldc doesn't decrease this overhead at all actually.
 Kind regards
 André
It's still a huge improvement. I'll take another look at the remaining overhead over the weekend.
Dec 06 2019
parent Seb <seb wilzba.ch> writes:
On Friday, 6 December 2019 at 08:51:03 UTC, mipri wrote:
 On Friday, 6 December 2019 at 07:47:33 UTC, Andre Pany wrote:
 [...]
It's calling dmd on every run, even with your patch: 1575620540.004204 execve("/usr/bin/dmd", ["dmd", "-quiet", "-c", "-o-", "-v", "long /tmp filename.d"] 1575620540.204598 execve("long tmp filename" ... but that seems to be probePlatform, which stopWatch says only takes 12ms. Statting the generated file shows that it's not getting touched, so it's definitely not getting recompiled.
 [...]
dmd, but a release build with ldc doesn't decrease this overhead at all actually.
 [...]
It's still a huge improvement. I'll take another look at the remaining overhead over the weekend.
Awesome! FYI: the platform probe could either be cached or the -Xi=compilerInfo -Xf=- info API which has been added in 2.079 for this purpose could be used. Details: https://github.com/dlang/dub/issues/1317
Dec 06 2019
prev sibling parent wjoe <invalid example.com> writes:
On Tuesday, 3 December 2019 at 16:33:32 UTC, Petar Kirov 
[ZombineDev] wrote:
 Dub is supposed to cache every build artifact, but for some 
 reason the `isUpToDate` function returns `false` for a 
 single-file package I am testing with.

 BTW, on what OS and hardware are you testing?
 I am on Linux with an SSD, so this may explain why I get faster 
 results than you.

 I'll look into the problem with Dub's `isUpToDate`.
First thing that comes to mind is that you mount your SSD with nomtime/noatime.
Dec 09 2019
prev sibling parent reply bachmeier <no spam.net> writes:
On Tuesday, 3 December 2019 at 09:10:16 UTC, Petar Kirov 
[ZombineDev] wrote:

 Instead of bashing it, I think this discussion could be more 
 productive if you shared your criticism in a constrictive way.
There have been many, many long threads on this already.
Dec 03 2019
parent reply mipri <mipri minimaltype.com> writes:
On Tuesday, 3 December 2019 at 13:56:33 UTC, bachmeier wrote:
 On Tuesday, 3 December 2019 at 09:10:16 UTC, Petar Kirov 
 [ZombineDev] wrote:

 Instead of bashing it, I think this discussion could be more 
 productive if you shared your criticism in a constrictive way.
There have been many, many long threads on this already.
And it is merely technically *possible* to receive them. It's not easy, or convenient, or definitely going to happen. The retrieved conversation is not necessarily even useful to someone capable of taking action on any matters raised but not capable of telling, without unrelated efforts, which of the matters raised is even still relevant, or not obviated by later (as yet unearthed) conversation. If you don't put your words in a wiki or an issue you should better treat the conversation as if it happened in person and your words were lost to the wind. Teams of archaeologists are not available.
Dec 03 2019
parent bachmeier <no spam.net> writes:
On Tuesday, 3 December 2019 at 14:45:47 UTC, mipri wrote:
 On Tuesday, 3 December 2019 at 13:56:33 UTC, bachmeier wrote:
 On Tuesday, 3 December 2019 at 09:10:16 UTC, Petar Kirov 
 [ZombineDev] wrote:

 Instead of bashing it, I think this discussion could be more 
 productive if you shared your criticism in a constrictive way.
There have been many, many long threads on this already.
And it is merely technically *possible* to receive them. It's not easy, or convenient, or definitely going to happen. The retrieved conversation is not necessarily even useful to someone capable of taking action on any matters raised but not capable of telling, without unrelated efforts, which of the matters raised is even still relevant, or not obviated by later (as yet unearthed) conversation. If you don't put your words in a wiki or an issue you should better treat the conversation as if it happened in person and your words were lost to the wind. Teams of archaeologists are not available.
Sounds good.
Dec 03 2019
prev sibling parent reply Seb <seb wilzba.ch> writes:
On Monday, 2 December 2019 at 18:48:41 UTC, aliak wrote:
 Like these in rund? 
 https://github.com/dragon-lang/rund#source-compiler-directives

 Is that something that dlang would be open to for rdmd?

 Cheers
 - Ali
Why not replace rdmd with rund? RDMD is not actively developed anymore (last commit was 1 1/2 years ago [1]) and it's still twice as slow as rund as the -i upgrade PRs were rejected (see e.g. [2]). [1] https://github.com/dlang/tools/commits/master/rdmd.d [2] https://github.com/dlang/tools/pull/292
Dec 02 2019
parent reply aliak <something something.com> writes:
On Monday, 2 December 2019 at 19:44:53 UTC, Seb wrote:
 On Monday, 2 December 2019 at 18:48:41 UTC, aliak wrote:
 Like these in rund? 
 https://github.com/dragon-lang/rund#source-compiler-directives

 Is that something that dlang would be open to for rdmd?

 Cheers
 - Ali
Why not replace rdmd with rund? RDMD is not actively developed anymore (last commit was 1 1/2 years ago [1]) and it's still twice as slow as rund as the -i upgrade PRs were rejected (see e.g. [2]). [1] https://github.com/dlang/tools/commits/master/rdmd.d [2] https://github.com/dlang/tools/pull/292
Hmm, there seems there's a bit too much resistance indeed :/ Why is that? THere's so much you can do with rdmd. The only reason for rdmd really is that it comes with a default installation on osx. Asking people to install the dmd package via osx is already some friction, but then telling them to do rund as well.... well basically we just go to python. I could maybe make a brew out of rund though ... 🤔
Dec 02 2019
parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Monday, 2 December 2019 at 20:13:24 UTC, aliak wrote:
 On Monday, 2 December 2019 at 19:44:53 UTC, Seb wrote:
 [...]
Hmm, there seems there's a bit too much resistance indeed :/ Why is that? THere's so much you can do with rdmd. The only reason for rdmd really is that it comes with a default installation on osx. Asking people to install the dmd package via osx is already some friction, but then telling them to do rund as well.... well basically we just go to python. I could maybe make a brew out of rund though ... 🤔
It's not a big deal if D tools are in separate packages. I added rund to the nix package manager already, adding it to brew should be fine.
Dec 02 2019
parent reply aliak <something something.com> writes:
On Monday, 2 December 2019 at 20:52:06 UTC, Jonathan Marler wrote:
 On Monday, 2 December 2019 at 20:13:24 UTC, aliak wrote:
 On Monday, 2 December 2019 at 19:44:53 UTC, Seb wrote:
 [...]
Hmm, there seems there's a bit too much resistance indeed :/ Why is that? THere's so much you can do with rdmd. The only reason for rdmd really is that it comes with a default installation on osx. Asking people to install the dmd package via osx is already some friction, but then telling them to do rund as well.... well basically we just go to python. I could maybe make a brew out of rund though ... 🤔
It's not a big deal if D tools are in separate packages. I added rund to the nix package manager already, adding it to brew should be fine.
I'm trying: https://github.com/Homebrew/homebrew-core/pull/47501 but apparently formulas recommend github repos with X stars and Y forks. So there's a chance it may be rejected. Let's see! 🤞
Dec 04 2019
parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Wednesday, 4 December 2019 at 23:32:04 UTC, aliak wrote:
 On Monday, 2 December 2019 at 20:52:06 UTC, Jonathan Marler 
 wrote:
 On Monday, 2 December 2019 at 20:13:24 UTC, aliak wrote:
 On Monday, 2 December 2019 at 19:44:53 UTC, Seb wrote:
 [...]
Hmm, there seems there's a bit too much resistance indeed :/ Why is that? THere's so much you can do with rdmd. The only reason for rdmd really is that it comes with a default installation on osx. Asking people to install the dmd package via osx is already some friction, but then telling them to do rund as well.... well basically we just go to python. I could maybe make a brew out of rund though ... 🤔
It's not a big deal if D tools are in separate packages. I added rund to the nix package manager already, adding it to brew should be fine.
I'm trying: https://github.com/Homebrew/homebrew-core/pull/47501 but apparently formulas recommend github repos with X stars and Y forks. So there's a chance it may be rejected. Let's see! 🤞
Looks like they require > 30 forks, > 30 watchers and > 75 stars. A reasonable requirement. rund has 1 fork, 2 watchers and 21 stars, so we'll have to wait and see if it gets more popular. To everyone, if you want to see rund in your package manager of choice, show your support by starring and/or watching it on github: https://github.com/dragon-lang/rund
Dec 04 2019
parent aliak <something something.com> writes:
On Thursday, 5 December 2019 at 02:34:17 UTC, Jonathan Marler 
wrote:
 On Wednesday, 4 December 2019 at 23:32:04 UTC, aliak wrote:
 On Monday, 2 December 2019 at 20:52:06 UTC, Jonathan Marler 
 wrote:
 [...]
I'm trying: https://github.com/Homebrew/homebrew-core/pull/47501 but apparently formulas recommend github repos with X stars and Y forks. So there's a chance it may be rejected. Let's see! 🤞
Looks like they require > 30 forks, > 30 watchers and > 75 stars. A reasonable requirement. rund has 1 fork, 2 watchers and 21 stars, so we'll have to wait and see if it gets more popular. To everyone, if you want to see rund in your package manager of choice, show your support by starring and/or watching it on github: https://github.com/dragon-lang/rund
Added my own tap if anyone wants it on osx+brew. brew install aliak00/dtap/rund or Or brew tap aliak00/dtap and then brew install rund
Dec 05 2019