www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Need advice on variable expansion behavior

Hi all,

Mathias LANG and me have been working on fixing a regression in 
Dub's variable expansion implementation [1]. Background:

In order to support use cases where a variable is used as a 
prefix or suffix in an identifier-like strings (e.g. 
build/${DUB_PACKAGE}_${DUB_PLATFORM}_${DUB_ARCH}), pull request 
[2] added support for syntax ${var}, in addition to the existing 
$var syntax.

Unfortunately that PR broke dollar sign escaping ($$) which was 
not caught by neither Dub's own unit tests, integration tests, 
nor any of the 46 Dub packages tested on Buildkite [4] (our CI 
for testing each pull request to dmd/druntime/phobos/tools/dub).

However, while testing the fix that Mathias originally wrote I 
bumped into a couple of edge cases and I realized the expected 
behavior is not thoroughly documented. As per the docs [4]:


 Inside of build setting values, it is possible to use variables 
 using dollar notation. Any variable not matching a predefined 
 name will be taken from the program environment. To denote a 
 literal dollar sign, use $$.


Should dollar signs not followed by a variable identifier be left 
as they are? What about escaped dollar signs ($$), or even $$$, 
$$$$, etc.? And what about $$$var?

For the purposes of this discussion, if we assume that each 
variable $var1 or ${var2} will be replaced by <VAR1> and <VAR2> 
respectively, which of the following outputs is the correct for 
the given input?

Source code:
https://run.dlang.io/gist/ZombineDev/eaf7076a7b83a556fac0c058b0525270?compiler=dmd


 Input:
 $ $$ $$$ $$$$ $$$$$ $$$$$$ 
 asd$$var1$var2$var3${var4}${var5}$${var6}$$var7$${var8} a$var9 
 b$$var10 c$$$var11 d$$$$var12 e$$$$$var13 f$$$$$$var14 
 g$$$$$$$var15 h$$$$$$$$var16 $$&$!$$?$$$$var18


 $ $$ $$$ $$$$ $$$$$ $$$$$$ 
 asd$var1<VAR2><VAR3><VAR4><VAR5>${var6}$var7${var8} a<VAR9> 
 b$var10 c$$var11 d$$$var12 e$$$$var13 f$$$$$var14 g$$$$$$var15 
 h$$$$$$$var16 $$&$!$$?$$$var18
 

 $ $ $$ $$ $$$ $$$ 
 asd$var1<VAR2><VAR3><VAR4><VAR5>${var6}$var7${var8} a<VAR9> 
 b$var10 c$<VAR11> d$$var12 e$$<VAR13> f$$$var14 g$$$<VAR15> 
 h$$$$var16 $&$!$?$$var18
 

 $ $ $$ $$ $$$ $$$ 
 asd$var1<VAR2>$var3<VAR4>${var5}${var6}$var7${var8} a<VAR9> 
 b$var10 c$$var11 d$$var12 e$$$var13 f$$$var14 g$$$$var15 
 h$$$$var16 $&$!$?$$var18
 
 Martin:
 $ $$ $$$ $$$$ $$$$$ $$$$$$ 
 asd$<VAR1><VAR2><VAR3><VAR4><VAR5>$<VAR6>$<VAR7>$<VAR8> a<VAR9> 
 b$<VAR10> c$$<VAR11> d$$$<VAR12> e$$$$<VAR13> f$$$$$<VAR14> 
 g$$$$$$<VAR15> h$$$$$$$<VAR16> $$&$!$$?$$$<VAR18>
 
 Original:
 <> $ $<> $$ $$<> $$$ 
 asd$var1<VAR2><VAR3><>{var4}<>{var5}${var6}$var7${var8} a<VAR9> 
 b$var10 c$<VAR11> d$$var12 e$$<VAR13> f$$$var14 g$$$<VAR15> 
 h$$$$var16 $&<>!$?$$var18
 
 Other: ???


Cheers,
Petar

[1]: https://github.com/dlang/dub/pull/1552
[2]: https://github.com/dlang/dub/pull/1392
[3]: 
https://gist.github.com/ZombineDev/fd79c4ffd58dbe9792d799f66b7f9434
[4]: 
https://code.dlang.org/package-format?lang=json#environment-variables
Sep 09 2018