www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - How to create compile-time container?

reply Andrey Zherikov <andrey.zherikov gmail.com> writes:
On a low level, I want something like this code to work:
     string[] arr;      // this can be any suitable type

     arr ~= "a";        // data is compile-time constant

     enum f = arr[0];   // fails with "Error: variable arr cannot 
be read at compile time"
     enum b = arr[$-1]; // fails with "Error: variable arr cannot 
be read at compile time"

More generic description: I'm looking for a way to have a 
container (or any other type) that I can manipulate during 
compile time. Operations needed are similar to stack 
functionality: get first/last element, push_back and pop_back.

How can I do that?
Aug 31 2020
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 31 August 2020 at 20:39:10 UTC, Andrey Zherikov wrote:

 How can I do that?


You can use a normal string[] BUT it is only allowed to be 
modified inside its own function.

Then you assign that function to an enum or whatever.


string[] ctGenerate() {
    string[] list;
    list ~= "stuff";
    return list;
}

enum list = ctGenerate();


That's all allowed. But CTFE is now allowed to read or modify 
anything outside its own function; you can't have two separate 
function calls build up a shared list (unless you can somehow 
call them both together like `enum list = ctGenerate() ~ 
other_thing();`
Aug 31 2020
parent reply Andrey Zherikov <andrey.zherikov gmail.com> writes:
On Monday, 31 August 2020 at 20:44:16 UTC, Adam D. Ruppe wrote:

 On Monday, 31 August 2020 at 20:39:10 UTC, Andrey Zherikov 
 wrote:

 How can I do that?


 You can use a normal string[] BUT it is only allowed to be 
 modified inside its own function.

 Then you assign that function to an enum or whatever.


 string[] ctGenerate() {
    string[] list;
    list ~= "stuff";
    return list;
 }

 enum list = ctGenerate();


 That's all allowed. But CTFE is now allowed to read or modify 
 anything outside its own function; you can't have two separate 
 function calls build up a shared list (unless you can somehow 
 call them both together like `enum list = ctGenerate() ~ 
 other_thing();`



The thing I'm trying to implement is: I have a function 
foo(string s)() and some "state"; this function should override 
this "state" (using "s" param) for all code within this function 
(note that code can execute other modules that can refer to the 
same "state"). The problem is that I need this overridden "state" 
to be compile-time constant to be used in mixin. Any ideas how I 
can do this?
Sep 01 2020
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 9/1/20 2:19 PM, Andrey Zherikov wrote:

 On Monday, 31 August 2020 at 20:44:16 UTC, Adam D. Ruppe wrote:

 On Monday, 31 August 2020 at 20:39:10 UTC, Andrey Zherikov wrote:

 How can I do that?


 You can use a normal string[] BUT it is only allowed to be modified 
 inside its own function.

 Then you assign that function to an enum or whatever.


 string[] ctGenerate() {
    string[] list;
    list ~= "stuff";
    return list;
 }

 enum list = ctGenerate();


 That's all allowed. But CTFE is now allowed to read or modify anything 
 outside its own function; you can't have two separate function calls 
 build up a shared list (unless you can somehow call them both together 
 like `enum list = ctGenerate() ~ other_thing();`


 
 
 The thing I'm trying to implement is: I have a function foo(string s)() 
 and some "state"; this function should override this "state" (using "s" 
 param) for all code within this function (note that code can execute 
 other modules that can refer to the same "state"). The problem is that I 
 need this overridden "state" to be compile-time constant to be used in 
 mixin. Any ideas how I can do this?


string overrideState(string s)
{
    // work your magic here, it's normal D code!
}

void foo(string s)()
{
    mixin(overrideState(s));
}

-Steve
Sep 01 2020
parent reply Andrey Zherikov <andrey.zherikov gmail.com> writes:
On Tuesday, 1 September 2020 at 18:57:30 UTC, Steven 
Schveighoffer wrote:

 string overrideState(string s)
 {
    // work your magic here, it's normal D code!
 }

 void foo(string s)()
 {
    mixin(overrideState(s));
 }


Unfortunately this won't work if there is a function 'bar' in 
different module that calls 'foo':

void bar()
{
    foo!"bar";
}

void foo(string s)()
{
   mixin(overrideState(s));

   ...
   mixin("bar()");     // just for illustration
}
Sep 01 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 9/1/20 3:09 PM, Andrey Zherikov wrote:

 Unfortunately this won't work if there is a function 'bar' in different 
 module that calls 'foo':


You should post a full example you expect to work or not work, then we 
can discuss.

I think it should work (I've tried it), but there are several problems 
that could possibly happen with your code, and it's hard to tell what 
you mean by "won't work" with the incomplete code that you posted.

-Steve
Sep 01 2020
parent reply Andrey Zherikov <andrey.zherikov gmail.com> writes:
On Tuesday, 1 September 2020 at 19:38:29 UTC, Steven 
Schveighoffer wrote:

 On 9/1/20 3:09 PM, Andrey Zherikov wrote:

 Unfortunately this won't work if there is a function 'bar' in 
 different module that calls 'foo':


 You should post a full example you expect to work or not work, 
 then we can discuss.

 I think it should work (I've tried it), but there are several 
 problems that could possibly happen with your code, and it's 
 hard to tell what you mean by "won't work" with the incomplete 
 code that you posted.

 -Steve


Sorry for confusion. I'm trying to implement compile-time 
scripting and the idea is pretty simple: I have a script file and 
I want to convert it to D code during compilation. I've done some 
things but stuck at the point when script includes another script.

For simplicity, let's say that script file has commands (one per 
line) with syntax "<command><space><parameter>" and only two 
commands available= "msg" to print text and "include" to include 
another script.
Here is my code:
==============
void parseFile(string file)()
{
     enum script = import(file);
     mixin(parseScript(script));
}
string parseScript(string script)
{
     string code;

     foreach(line; script.lineSplitter())
     {
         auto idx = line.indexOf(' ');

         switch(line[0..idx])
         {
             case "msg"=
                 code ~= "writeln(\"" ~ line[idx+1..$] ~ "\");";
                 break;
             case "include"=
                 code ~= "parseFile!\"" ~ line[idx+1..$] ~ "\";";
                 break;
             default= break;
         }
     }
     return code;
}
void main()
{
     parseFile!"script";
}
==============
Everything works well until I have included scripts in 
subdirectories:
├── dir1
│   ├── dir2
│   │   └── script
│   └── script
└── script
Content:
============== script
msg hello
include dir1/script
============== dir1/script
msg hello from dir1
include dir2/script
============== dir1/dir2/script
msg hello from dir1/dir2
==============

Compilation fails with "Error: file `"dir2/script"` cannot be 
found or not in a path specified with `-J`" (I used simple dmd 
-J. -run parser.d) which is expected because parse* functions do 
not track the directory where the script is located.

In this simple example the issue can be fixed by passing path to 
script as a parameter to parseScript function. But this doesn't 
seem to be flexible and extendable solution because there can be 
other commands that might call parseFile indirectly (they can 
even be in other modules).

Theoretically this can be solved by doing something like this but 
it doesn't work because "static variable `paths` cannot be read 
at compile time":
==============
string[] paths;
void parseFile(string file)()
{
     enum path = paths.length > 0 ? buildPath(paths[$-1], 
file.dirName()) : file.dirName();

     paths ~= path;
     scope(exit) paths = paths[0..$-1];

     enum script = import(buildPath(path, file));
     mixin(parseScript(script));
}
==============
Note that the whole point is to do this parsing at compile time.
Sep 02 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 9/2/20 5:56 AM, Andrey Zherikov wrote:


 ==============
 Everything works well until I have included scripts in subdirectories:
 ├── dir1
 │   ├── dir2
 │   │   └── script
 │   └── script
 └── script
 Content:
 ============== script
 msg hello
 include dir1/script
 ============== dir1/script
 msg hello from dir1
 include dir2/script
 ============== dir1/dir2/script
 msg hello from dir1/dir2
 ==============
 
 Compilation fails with "Error: file `"dir2/script"` cannot be found or 
 not in a path specified with `-J`" (I used simple dmd -J. -run parser.d) 
 which is expected because parse* functions do not track the directory 
 where the script is located.
 
 In this simple example the issue can be fixed by passing path to script 
 as a parameter to parseScript function. But this doesn't seem to be 
 flexible and extendable solution because there can be other commands 
 that might call parseFile indirectly (they can even be in other modules).
 
 Theoretically this can be solved by doing something like this but it 
 doesn't work because "static variable `paths` cannot be read at compile 
 time":
 ==============
 string[] paths;
 void parseFile(string file)()
 {
      enum path = paths.length > 0 ? buildPath(paths[$-1], 
 file.dirName()) : file.dirName();
 
      paths ~= path;
      scope(exit) paths = paths[0..$-1];
 
      enum script = import(buildPath(path, file));
      mixin(parseScript(script));
 }
 ==============
 Note that the whole point is to do this parsing at compile time.



OK, NOW I see where your code is coming from. The issue is that you need 
the directory of the script imported to be the "local directory". Your 
solution will not work -- you can't mixin code that is not available at 
compile time.

Here is what I would do instead:

string parseScript(string filename, string script)
{
     string code;
     string base = dirName(filename);
     if(base[$-1] != '/') base ~= '/';
     foreach(line; script.lineSplitter())
     {
         auto idx = line.indexOf(' ');

         switch(line[0..idx])
         {
             case "msg":
                 code ~= "writeln(\"" ~ line[idx+1..$] ~ "\");";
                 break;
             case "include":
             {
                 code ~= `parseFile!"`;
                 string importfile = line[idx+1 .. $];
                 if(!importfile.startsWith('/')) // relative path
                      code ~= base;
                 code ~= importfile ~ `";`;
                 break;
             }
             default: break;
         }
     }
     return code;
}

And pass the filename to this function in addition to the script source.

-Steve
Sep 02 2020
parent reply Andrey Zherikov <andrey.zherikov gmail.com> writes:
On Wednesday, 2 September 2020 at 14:38:30 UTC, Steven 
Schveighoffer wrote:

 Here is what I would do instead:


Thanks, I'll try it.
Sep 02 2020
parent Andrey Zherikov <andrey.zherikov gmail.com> writes:
On Wednesday, 2 September 2020 at 17:40:55 UTC, Andrey Zherikov 
wrote:

 On Wednesday, 2 September 2020 at 14:38:30 UTC, Steven 
 Schveighoffer wrote:

 Here is what I would do instead:


 Thanks, I'll try it.


This actually works! Thanks a lot!
Sep 02 2020
prev sibling parent Andrey Zherikov <andrey.zherikov gmail.com> writes:
On Tuesday, 1 September 2020 at 18:19:55 UTC, Andrey Zherikov 
wrote:

 The thing I'm trying to implement is: I have a function 
 foo(string s)() and some "state"; this function should override 
 this "state" (using "s" param) for all code within this 
 function (note that code can execute other modules that can 
 refer to the same "state"). The problem is that I need this 
 overridden "state" to be compile-time constant to be used in 
 mixin. Any ideas how I can do this?


In regular code it should look like this:

private string[] state;

string getCurrentState()
{
   return state[$-1];
}

void foo(string s)
{
   state ~= s;
   scope(exit) state = state[0..$-1];

   ...
}
Sep 01 2020
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Aug 31, 2020 at 08:39:10PM +0000, Andrey Zherikov via
Digitalmars-d-learn wrote:

 On a low level, I want something like this code to work:
     string[] arr;      // this can be any suitable type
 
     arr ~= "a";        // data is compile-time constant
 
     enum f = arr[0];   // fails with "Error: variable arr cannot be read at
 compile time"
     enum b = arr[$-1]; // fails with "Error: variable arr cannot be read at
 compile time"
 
 More generic description: I'm looking for a way to have a container
 (or any other type) that I can manipulate during compile time.
 Operations needed are similar to stack functionality: get first/last
 element, push_back and pop_back.
 
 How can I do that?


First, read this article:

	https://wiki.dlang.org/User:Quickfur/Compile-time_vs._compile-time

Secondly, to answer your question: write regular ("runtime") code for
manipulating your data (also regular "runtime" data), and simply run the
function during CTFE. Almost 90% of the language is usable during CTFE,
so that's all you really need.


T

-- 
Маленькие детки - маленькие бедки.
Aug 31 2020