www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - jai-like CTFE string formating

reply Stefan Koch <uplink.coder googlemail.com> writes:
Hi Guys,

I've just implemented a subset of the std.format functionality.
In the same style that Johnathan Blow uses for JAI.

It's about 10x faster then using std.format and uses much less 
memory :)

the follwing code takes over 250 ms to compile :
{
     import std.format;
     pragma(msg, format("Hello %s %s %s %s %s %s %s %s %s", " I ", 
  "just", " have", " to", " concat",  " a lot", " of", " strings 
...", 9));
}
Whereas the following alternative takes 20 ms :
{
     import ctfe_utils;
     pragma(msg, format_jai("Hello % % % % % % % % %", " I ", " 
just", " have" , " to", " concat", " a lot", " of", " strings 
...", 9));
}

see for yourself:
https://www.youtube.com/watch?v=T0BJxdt61RY
Aug 12 2017
next sibling parent tetyys <tetyys tetyys.com> writes:
On Saturday, 12 August 2017 at 11:47:10 UTC, Stefan Koch wrote:

 [...]


very nice
Aug 12 2017
prev sibling next sibling parent reply ixid <nuaccount gmail.com> writes:
On Saturday, 12 August 2017 at 11:47:10 UTC, Stefan Koch wrote:

 Whereas the following alternative takes 20 ms :
 {
     import ctfe_utils;
     pragma(msg, format_jai("Hello % % % % % % % % %", " I ", " 
 just", " have" , " to", " concat", " a lot", " of", " strings 
 ...", 9));
 }

 see for yourself:
 https://www.youtube.com/watch?v=T0BJxdt61RY


Could we not improve the syntax to something like:

      pragma(msg, format_jai("Hello %9", " I ", "
  just", " have" , " to", " concat", " a lot", " of", " strings
  ..."));
  }

Or whatever is closest that doesn't step on the current print 
syntax.
Aug 12 2017
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 12 August 2017 at 13:19:12 UTC, ixid wrote:

 On Saturday, 12 August 2017 at 11:47:10 UTC, Stefan Koch wrote:

 Whereas the following alternative takes 20 ms :
 {
     import ctfe_utils;
     pragma(msg, format_jai("Hello % % % % % % % % %", " I ", " 
 just", " have" , " to", " concat", " a lot", " of", " strings 
 ...", 9));
 }

 see for yourself:
 https://www.youtube.com/watch?v=T0BJxdt61RY


 Could we not improve the syntax to something like:

      pragma(msg, format_jai("Hello %9", " I ", "
  just", " have" , " to", " concat", " a lot", " of", " strings
  ..."));
  }

 Or whatever is closest that doesn't step on the current print 
 syntax.


The point of format is that you want to replace the % my the 
argument and then write more static text.

"The Parameter '%' was null and it should not be"
"Rule '%' was not found in catalog '%' is possible the closest 
match"
Aug 12 2017
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 13 August 2017 at 00:15, Stefan Koch via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Saturday, 12 August 2017 at 13:19:12 UTC, ixid wrote:


 On Saturday, 12 August 2017 at 11:47:10 UTC, Stefan Koch wrote:


 Whereas the following alternative takes 20 ms :
 {
     import ctfe_utils;
     pragma(msg, format_jai("Hello % % % % % % % % %", " I ", " just", "
 have" , " to", " concat", " a lot", " of", " strings ...", 9));
 }

 see for yourself:
 https://www.youtube.com/watch?v=T0BJxdt61RY


 Could we not improve the syntax to something like:

      pragma(msg, format_jai("Hello %9", " I ", "
  just", " have" , " to", " concat", " a lot", " of", " strings
  ..."));
  }

 Or whatever is closest that doesn't step on the current print syntax.


 The point of format is that you want to replace the % my the argument and
 then write more static text.

 "The Parameter '%' was null and it should not be"
 "Rule '%' was not found in catalog '%' is possible the closest match"


If you're keen to introduce a new function, I'd strongly suggest changing
to {1} {2} {3}, or %1 %2 %3, format/printf functions where you don't supply
the place index are useless a lot of the time and need to be rewritten,
which is a tedious task.
Aug 12 2017
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 13 August 2017 at 00:42:08 UTC, Manu wrote:

 On 13 August 2017 at 00:15, Stefan Koch via Digitalmars-d <

 [ ... ]


 If you're keen to introduce a new function, I'd strongly 
 suggest changing to {1} {2} {3}, or %1 %2 %3, format/printf 
 functions where you don't supply the place index are useless a 
 lot of the time and need to be rewritten, which is a tedious 
 task.


I am not too keen introducing a new function, given that it most 
likely won't be accepted into phobos.

However your suggestion is indeed a good suggestion.
The check would then see if the number of args are equal to the 
max index.
And it would check that all indices from 0 to maxIdx are used at 
least once.

Thanks,
Stefan
Aug 12 2017
parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 13 August 2017 at 12:07, Stefan Koch via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Sunday, 13 August 2017 at 00:42:08 UTC, Manu wrote:


 On 13 August 2017 at 00:15, Stefan Koch via Digitalmars-d <


 [ ... ]


 If you're keen to introduce a new function, I'd strongly suggest changing
 to {1} {2} {3}, or %1 %2 %3, format/printf functions where you don't supply
 the place index are useless a lot of the time and need to be rewritten,
 which is a tedious task.


 I am not too keen introducing a new function, given that it most likely
 won't be accepted into phobos.

 However your suggestion is indeed a good suggestion.
 The check would then see if the number of args are equal to the max index.
 And it would check that all indices from 0 to maxIdx are used at least
 once.


No, it's not always the case that a format string uses every argument.
Format strings are usually taken from the translation table. Different
languages have different word orders (which is why placeholders need to
accept an arg index), and occasionally, some translations don't use all the
args because the sentence structure doesn't make sense, or excess args are
supplied because one language requires a string that others don't use.
Aug 13 2017
prev sibling next sibling parent reply =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig+d outerproduct.org> writes:
Am 12.08.2017 um 13:47 schrieb Stefan Koch:

 Hi Guys,
 
 I've just implemented a subset of the std.format functionality.
 In the same style that Johnathan Blow uses for JAI.
 
 It's about 10x faster then using std.format and uses much less memory :)
 
 the follwing code takes over 250 ms to compile :
 {
      import std.format;
      pragma(msg, format("Hello %s %s %s %s %s %s %s %s %s", " I ", 
   "just", " have", " to", " concat",  " a lot", " of", " strings ...", 9));
 }
 Whereas the following alternative takes 20 ms :
 {
      import ctfe_utils;
      pragma(msg, format_jai("Hello % % % % % % % % %", " I ", " just", " 
 have" , " to", " concat", " a lot", " of", " strings ...", 9));
 }
 
 see for yourself:
 https://www.youtube.com/watch?v=T0BJxdt61RY


I was a bit shocked by this number and performed a little test with 
multiple calls to format. Fortunately only the first one takes that long 
to compile. For 500 different calls I got about 2ms per call on Windows 
(which might get slowed down somewhat by the slow terminal output):

     import std.format;
     template T(size_t i) {
       static if (i > 0) {
         pragma(msg, format("Bye %s %s %s %s %s %s %s %s %s %s",
           i, " I ",  "just", " have", " to", " concat",  " a lot",
           " of", " strings ...", 9));
         enum T = T!(i-1);
       } else enum T = 0;
     }
     enum test = T!500;
Aug 13 2017
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 13 August 2017 at 18:20:15 UTC, Sönke Ludwig wrote:

 Am 12.08.2017 um 13:47 schrieb Stefan Koch:

 Hi Guys,
 
 I've just implemented a subset of the std.format functionality.
 In the same style that Johnathan Blow uses for JAI.
 
 It's about 10x faster then using std.format and uses much less 
 memory :)
 
 the follwing code takes over 250 ms to compile :
 {
      import std.format;
      pragma(msg, format("Hello %s %s %s %s %s %s %s %s %s", " 
 I ",
   "just", " have", " to", " concat",  " a lot", " of", " 
 strings ...", 9));
 }
 Whereas the following alternative takes 20 ms :
 {
      import ctfe_utils;
      pragma(msg, format_jai("Hello % % % % % % % % %", " I ", 
 " just", " have" , " to", " concat", " a lot", " of", " 
 strings ...", 9));
 }
 
 see for yourself:
 https://www.youtube.com/watch?v=T0BJxdt61RY


 I was a bit shocked by this number and performed a little test 
 with multiple calls to format. Fortunately only the first one 
 takes that long to compile. For 500 different calls I got about 
 2ms per call on Windows (which might get slowed down somewhat 
 by the slow terminal output):

     import std.format;
     template T(size_t i) {
       static if (i > 0) {
         pragma(msg, format("Bye %s %s %s %s %s %s %s %s %s %s",
           i, " I ",  "just", " have", " to", " concat",  " a 
 lot",
           " of", " strings ...", 9));
         enum T = T!(i-1);
       } else enum T = 0;
     }
     enum test = T!500;


Wouldn't you say that 2ms is still a pretty long time for a 
function call ?
Aug 13 2017
parent =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig+d outerproduct.org> writes:
Am 13.08.2017 um 20:25 schrieb Stefan Koch:

 On Sunday, 13 August 2017 at 18:20:15 UTC, Sönke Ludwig wrote:

 I was a bit shocked by this number and performed a little test with 
 multiple calls to format. Fortunately only the first one takes that 
 long to compile. For 500 different calls I got about 2ms per call on 
 Windows (which might get slowed down somewhat by the slow terminal 
 output):

     import std.format;
     template T(size_t i) {
       static if (i > 0) {
         pragma(msg, format("Bye %s %s %s %s %s %s %s %s %s %s",
           i, " I ",  "just", " have", " to", " concat",  " a lot",
           " of", " strings ...", 9));
         enum T = T!(i-1);
       } else enum T = 0;
     }
     enum test = T!500;


 
 Wouldn't you say that 2ms is still a pretty long time for a function call ?
 


For this particular function call, yes, but still a lot less scary ;) To 
get some better numbers, can you do a side by side comparison of your 
two versions both executed a few hundred times? If the simplified 
version still turns out considerably faster then there may be some low 
hanging fruits in vibe.d's web/REST modules.
Aug 13 2017
prev sibling parent reply Jerry <hurricane hereiam.com> writes:
Seems like it'd be a good idea to pre compute all of phobos for 
compile time computations, as they should be changing. That would 
drastically reduce using any of phobos for compile time 
computation.
Aug 13 2017
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 13 August 2017 at 19:47:37 UTC, Jerry wrote:

 Seems like it'd be a good idea to pre compute all of phobos for 
 compile time computations, as they should be changing. That 
 would drastically reduce using any of phobos for compile time 
 computation.


You cannot do that.
The point of template code is that the user can change it from 
the outside.
There is no way to precompute the reaction to things that are yet 
to be realized.

The way compile time is saved here is by limiting the flexibility.
Aug 13 2017
parent Jerry <hurricane hereiam.com> writes:
On Sunday, 13 August 2017 at 19:51:31 UTC, Stefan Koch wrote:

 On Sunday, 13 August 2017 at 19:47:37 UTC, Jerry wrote:

 Seems like it'd be a good idea to pre compute all of phobos 
 for compile time computations, as they should be changing. 
 That would drastically reduce using any of phobos for compile 
 time computation.


 You cannot do that.
 The point of template code is that the user can change it from 
 the outside.
 There is no way to precompute the reaction to things that are 
 yet to be realized.

 The way compile time is saved here is by limiting the 
 flexibility.


There are ways to do it, not having to parse the code would make 
it faster on its own. Compiling 500 source files individually 
that starts and stops DMD is drastically slower than putting the 
500 source files together and running DMD once. The tests that 
are run for DMD for example could be made to run much faster.
Aug 19 2017