www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Multiline string literal improvements

reply Igor <stojkovic.igor gmail.com> writes:
D has a very nice feature of token strings:
string a = q{
                looksLikeCode();
             };

It is useful in writing mixins and to have syntax highlighting in 
editors. Although I like it, it is not something I ever felt like 
missing in other languages. What I do always miss are these two 
options:

1. Have something like this:

string a = |q{
                  firstLine();
                  if (cond) {
                      secondLine()
                  }
               };

mean count the number of whitespace characters at the start of a 
first new line of string literal and then strip up to that many 
whitespace characters from the start of each line.

2. If we just put for example "-" instead of "|" in above example 
have that mean: replace all whitespace with a single space in 
following string literal.

I think it is clear why would these be useful but if you want me 
I can add a few examples. This would not make any breaking 
changes to the language and it should be possible to simply 
implement it wholly in the lexer.

So what do think?
Oct 10 2017
parent reply captaindet <2krnk gmx.net> writes:
 string a = |q{
                   firstLine();
                   if (cond) {
                       secondLine()
                   }
                };


you could write your own string processing function according to your 
needs to filter the code string, and use it like
string a = inject(q{...})  			//or
string a = inject!(formatOpts)(q{...})

i have done this myself and also included positional argument formatting 
to my liking, optimized for CT code generation. don't have the code at 
my hands ATM though. could post it later if you are interested.

/det
Oct 10 2017
parent reply sarn <sarn theartofmachinery.com> writes:
On Tuesday, 10 October 2017 at 21:38:41 UTC, captaindet wrote:

 string a = |q{
                   firstLine();
                   if (cond) {
                       secondLine()
                   }
                };


 you could write your own string processing function according 
 to your needs


FWIW, that's the solution in Python:
https://docs.python.org/release/3.6.3/library/textwrap.html#textwrap.dedent

Works even better in D because it can run at compile time.
Oct 10 2017
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/10/2017 3:16 PM, sarn wrote:

 Works even better in D because it can run at compile time.


Yes, I see no need for a language feature what can be easily and far more 
flexibly done with a regular function - especially since what |q{ and -q{ do 
gives no clue from the syntax.
Oct 11 2017
next sibling parent reply Igor <stojkovic.igor gmail.com> writes:
On Wednesday, 11 October 2017 at 08:35:51 UTC, Walter Bright 
wrote:

 On 10/10/2017 3:16 PM, sarn wrote:

 Works even better in D because it can run at compile time.


 Yes, I see no need for a language feature what can be easily 
 and far more flexibly done with a regular function - especially 
 since what |q{ and -q{ do gives no clue from the syntax.


You are right. My mind is just still not used to the power of D 
templates so I didn't think of this. On the other hand that is 
why D is still making me say "WOW!" on a regular basis :).

Just to confirm I understand, for example the following would 
give me compile time stripping of white space:

template stripws(string l) {
     enum stripws = l.replaceAll(regex("\s+", "g"), " ");
}

string variable = stripws(q{
    whatever and    ever;
});

And I would get variable to be equal to " whatever and ever; ". 
Right?
Oct 11 2017
parent reply Meta <jared771 gmail.com> writes:
On Wednesday, 11 October 2017 at 09:56:52 UTC, Igor wrote:

 On Wednesday, 11 October 2017 at 08:35:51 UTC, Walter Bright 
 wrote:

 On 10/10/2017 3:16 PM, sarn wrote:

 Works even better in D because it can run at compile time.


 Yes, I see no need for a language feature what can be easily 
 and far more flexibly done with a regular function - 
 especially since what |q{ and -q{ do gives no clue from the 
 syntax.


 You are right. My mind is just still not used to the power of D 
 templates so I didn't think of this. On the other hand that is 
 why D is still making me say "WOW!" on a regular basis :).

 Just to confirm I understand, for example the following would 
 give me compile time stripping of white space:

 template stripws(string l) {
     enum stripws = l.replaceAll(regex("\s+", "g"), " ");
 }

 string variable = stripws(q{
    whatever and    ever;
 });

 And I would get variable to be equal to " whatever and ever; ". 
 Right?


Even better, you could write the same code that you would for 
doing this at runtime and it'll Just Work:

string variable = q{
     whatever and  ever;
}.replaceAll(regex(`\s+`, "g"), " ");
Oct 11 2017
parent reply Igor <stojkovic.igor gmail.com> writes:
On Wednesday, 11 October 2017 at 14:28:32 UTC, Meta wrote:

 On Wednesday, 11 October 2017 at 09:56:52 UTC, Igor wrote:

 On Wednesday, 11 October 2017 at 08:35:51 UTC, Walter Bright 
 wrote:

 On 10/10/2017 3:16 PM, sarn wrote:

 Works even better in D because it can run at compile time.


 Yes, I see no need for a language feature what can be easily 
 and far more flexibly done with a regular function - 
 especially since what |q{ and -q{ do gives no clue from the 
 syntax.


 You are right. My mind is just still not used to the power of 
 D templates so I didn't think of this. On the other hand that 
 is why D is still making me say "WOW!" on a regular basis :).

 Just to confirm I understand, for example the following would 
 give me compile time stripping of white space:

 template stripws(string l) {
     enum stripws = l.replaceAll(regex("\s+", "g"), " ");
 }

 string variable = stripws(q{
    whatever and    ever;
 });

 And I would get variable to be equal to " whatever and ever; 
 ". Right?


 Even better, you could write the same code that you would for 
 doing this at runtime and it'll Just Work:

 string variable = q{
     whatever and  ever;
 }.replaceAll(regex(`\s+`, "g"), " ");


I tried this but Disassembly view shows:

call std.regex.regex!string.regex
and
call std.regex.replaceAll!(string, char, 
std.regex.internal.ir.Regex!char).replaceAll

which means that replaceAll with regex is done at runtime, not 
compile time. Also when I just added enum in front of string 
variable then I got this:

Error: malloc cannot be interpreted at compile time, because it 
has no available source code
Oct 12 2017
parent reply Meta <jared771 gmail.com> writes:
On Thursday, 12 October 2017 at 08:08:17 UTC, Igor wrote:

 I tried this but Disassembly view shows:

 call std.regex.regex!string.regex
 and
 call std.regex.replaceAll!(string, char, 
 std.regex.internal.ir.Regex!char).replaceAll

 which means that replaceAll with regex is done at runtime, not 
 compile time. Also when I just added enum in front of string 
 variable then I got this:

 Error: malloc cannot be interpreted at compile time, because it 
 has no available source code


Hmm, you're right. I could've sworn that std.regex is 
CTFE-friendly but it looks like I was wrong. If it used the GC 
instead of malloc this would probably work.
Oct 12 2017
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On Thursday, 12 October 2017 at 16:59:46 UTC, Meta wrote:

 On Thursday, 12 October 2017 at 08:08:17 UTC, Igor wrote:

 I tried this but Disassembly view shows:




[snip]

 Hmm, you're right. I could've sworn that std.regex is 
 CTFE-friendly but it looks like I was wrong. If it used the GC 
 instead of malloc this would probably work.


Indeed it’s been an ongoing work to make regex match at CTFE. I 
considered peperring the code paths with __ctfe ? malloc ... : 
new[]
But it a lot of corner cases snd run-time optimized code path is 
already quite impenetrable.
Oct 12 2017
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2017-10-11 10:35, Walter Bright wrote:

 On 10/10/2017 3:16 PM, sarn wrote:

 Works even better in D because it can run at compile time.


 
 Yes, I see no need for a language feature what can be easily and far 
 more flexibly done with a regular function - especially since what |q{ 
 and -q{ do gives no clue from the syntax.


Unfortunately it doesn't work for the other multiline syntax:

void main()
{
     auto a = q"FOO
         int b = 3;
     FOO";
}

The above fails to compile [1]. The trailing FOO cannot be indented. 
This works:

void main()
{
     auto a = q"FOO
         int b = 3;
FOO";
}

Which in my opinion doesn't look as good as the first example. It gets 
worse if "a" is indented even more, because it's nested in a class, in a 
method, in an if statement and so on.

[1] main.d(3,14): Error: unterminated delimited string constant starting 
at main.d(3,15)

-- 
/Jacob Carlborg
Oct 12 2017
prev sibling parent reply Biotronic <simen.kjaras gmail.com> writes:
On Tuesday, 10 October 2017 at 22:16:00 UTC, sarn wrote:

 On Tuesday, 10 October 2017 at 21:38:41 UTC, captaindet wrote:

 string a = |q{
                   firstLine();
                   if (cond) {
                       secondLine()
                   }
                };


 you could write your own string processing function according 
 to your needs


 FWIW, that's the solution in Python:
 https://docs.python.org/release/3.6.3/library/textwrap.html#textwrap.dedent

 Works even better in D because it can run at compile time.


D version that works in CTFE:

import std.ascii : newline;

string dedent(string s, string newline = newline) {
     import std.string : strip, splitLines, front, join;
     import std.uni : isWhite;
     import std.array : array;
     import std.algorithm : until, startsWith;

     auto lines = s.strip().splitLines();
     if (lines.length == 0) return "";
     if (lines.length == 1) return lines[0];

     auto whitespace = lines[1].until!(a => !a.isWhite).array;

     foreach (ref line; lines) {
         if (line.startsWith(whitespace)) {
             line = line[whitespace.length..$];
         }
         // Throw if line doesn't start with correct amount of 
whitespace?
     }

     return lines.join(newline);
} unittest {
     assert(dedent("a") == "a");
     assert(dedent("a\n a") == "a"~newline~"a");
     string a = q{
                  firstLine();
                  if (cond) {
                      secondLine();
                  }
               };
     string b = "firstLine();"~newline~"if (cond) {"~newline~"    
secondLine();"~newline~"}";
     assert(a.dedent == b);
}

template dedent(string s, string newline = newline) {
     enum dedent = .dedent(s, newline);
} unittest {
     assert(dedent!("a\n a", "\n") == "a\na");
}

Treats all whitespace the same ('\t' == ' ' == MONGOLIAN VOWEL 
SEPARATOR), which might not be optimal, but I'm not gonna touch 
that can of worms.

--
   Biotronic
Oct 13 2017
parent Igor <stojkovic.igor gmail.com> writes:
On Friday, 13 October 2017 at 07:59:36 UTC, Biotronic wrote:

 D version that works in CTFE:


Thanks Biotronic! This is just what I had in mind.
Oct 13 2017