www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - various questions

reply Jason Spencer <spencer8 sbcglobal.net> writes:
I'm working on a program to do statistics on matrices of different sizes, and
I've run into a handful of situations where I just can't seem to find the
trick I need.  In general, I'm trying to make my functions work on static
arrays of the proper size, and template that up for the different sizes I need
to support.  I appeal to your experience and mercy on the various questions
below.  Any help on any point appreciated.

1.  How can I cast a single dimension dynamic array to a multi-dimension
static array?  I'm trying to do roughly the following:

   auto data = cast(float[100][100])std.file.read(fname, fsize);

which fails on the cast.  Is there any way to treat they memory returned
from read as a static array?  If I want to avoid the copy, am I relegated back
to pointers?  Is there a cast path through a pointer that would work?  I think
I'm essentially asking if I can make a reference to a static array and assign
that to the memory contained in a dynamic array.  I have a suspicion about the
answer....

2.  To work around 1., I've left one dimension of the cast floating.  This
means to bind both dimensions in my template args, I can no longer use
type parameter inference, so I have to explicitly instantiate the template
with the static size, but pass in a type that's dynamic in one dimension.  Is
there a way to deduce the equivalent static array type from the dynamic array?
(See code below).  How about just from a static array?  Even the following
fails:

   U foo(T: U[C][R], U, size_t C, size_t R)(T data, size_t c, size_t r)
   {
      return data[r][c];
   }

   int[4][5] data = [ [...] ];
   int bar = foo(data, 2, 3);

but

   int bar = foo!(int[4][5])(data, 2, 3);

works.  Why?

If I could solve that, then next I'd need to make this work:

   U foo(T: U[C][R], U, size_t C, size_t R)(U[C][] data, size_t c, size_t r)
   {
      return data[r][c];
   }

   int[4][] data = [ [...] ];
   int bar = foo(data, 2, 3);

Extra points for that one!


3.  I wanted to have a const static lookup table in a struct to map a string to
an
int.  I figured an Associative array would be a good way to index by string.
But I was not able to give a literal AA.  I had to put it in static this().  I
assume that's because AA's aren't really initialized by the compiler, so
they're not really literals.  They're just arguments to some run-time function
that actually initializes them.  Is that right?  If so, that's too bad--making
different rules for different array types makes them feel like they're not
/really/ part of the language.  Plus, having immutable lookups, fixed at
compile time, would be really useful.

4.  Lastly, compiling.  In src/main.d, I had "import data.stats;", and in
src/data/stats.d, I had "module data.stats;".  On the command line in src, I
figured
I'd just have to say "dmd main.d" and I'd be good to go.  Or maybe "dmd -I.
main.d" or "dmd -Idata main.d".  But I had to specify both files for the
compiler, as in "dmd main.d data/stats.d" to avoid undefined symbols from
stat.d at link time.  Is that right?  If so, how does dmd know where to get
phobos files from?  Is there a slicker way to do this with dmd?

Thanks,
Jason
Jul 28 2010
next sibling parent Jason Spencer <spencer8 sbcglobal.net> writes:
Forgot a couple of things:

- this is all using D2.047.

- Another question (in reference to part 2 before):  I'd like to
support about 4 base types and 5 or 6 different matrix sizes.  So
that's roughly 20 type combinations for my template.  But I decide
these based on command-line arguments at run time.  So I need some
slick way of mapping the run-time element size, row count, and column
count into a static array type to instantiate my template with.
Anybody have any good ideas how to make that table-driven or
something?

Thanks again,
Jason
Jul 28 2010
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Jason Spencer:

 1.  How can I cast a single dimension dynamic array to a multi-dimension
 static array?  I'm trying to do roughly the following:
    auto data = cast(float[100][100])std.file.read(fname, fsize);


I have tried a natural implementation, but it doesn't work and I don't know why
it doesn't work or how to fix it:

import std.file, std.stdio;
void main() {
    enum int N = 10;
    auto filename = "data.dat";
    auto data = slurp!(float[N][N])(filename, "%s")[0];
}


So I have written an ugly version:

import std.file, std.stdio, std.contracts;

void main() {
    enum int N = 10;
    alias float DataType;
    auto filename = "data.dat";

    // be careful with endiness
    enum size_t data_size = N * N * DataType.sizeof;
    void[] raw = std.file.read(filename, data_size);
    enforce(raw.length == data_size);
    DataType[N][N] data = void;
    data[0].ptr[0 .. N*N] = (cast(DataType[])raw)[];
}


But that creates dynamic arrays, copies them, etc. Ugly, unsafe and
inefficient. So better to have unsafe and efficient code, going back to C:

import std.contracts, std.c.stdio;

void main() {
    enum int N = 10;
    alias float DataType;
    auto filename = "data.dat";

    DataType[N][N] data = void;

    auto fin = fopen((filename ~ '\0').ptr, "rb");
    enforce(fin != null);

    // be careful with endiness
    size_t nread = fread(data[0].ptr, DataType.sizeof, N * N, fin);
    enforce(nread == N * N);

    fclose(fin);
}


In D fixed-sized nD arrays are contiguous, so I think that code is correct.
Be careful with floats endinaness.
Other people maybe can give you better solutions. But the point is that a
minimally useful Phobos2 must be able to read/write a binary basic type (like a
matrix) from disk with no extra copying and allocations and with a safe and
nice syntax.



 They're just arguments to some run-time function
 that actually initializes them.


Currently AAs are (mostly) run-time things. Things can change in future as CTFE
gets better.



 4.  Lastly, compiling.  In src/main.d, I had "import data.stats;", and in
 src/data/stats.d, I had "module data.stats;".  On the command line in src, I
figured
 I'd just have to say "dmd main.d" and I'd be good to go.  Or maybe "dmd -I.
 main.d" or "dmd -Idata main.d".  But I had to specify both files for the
 compiler, as in "dmd main.d data/stats.d" to avoid undefined symbols from
 stat.d at link time.  Is that right?  If so, how does dmd know where to get
 phobos files from?  Is there a slicker way to do this with dmd?


<sarcasm>Walter seems to think that Java and Python are too much handy on this,
and he thinks it's not the work of the compiler to look for the file it needs
by itself. Probably because this can become slow if a project has thousands of
source files. So to make things clean and tidy it's better to never make dmd
handy, even if your project is 2 files long. So you need to use an external
tool to solve this problem. I use one named "bud" not being able to use the
built-in one named rdmd, and Andrei didn't help me to learn how to use it and I
have not found docs about it. There are other tools good for larger projects
that of course are much harder to use.</sarcasm>  I have never appreciated all
this situation. Good luck.



 But I decide
 these based on command-line arguments at run time.  So I need some
 slick way of mapping the run-time element size, row count, and column
 count into a static array type to instantiate my template with.


A basic brutal way to do that is to use a switch, that switches according to
the runtime value to use the correctly compile-time instantiated template. If
the switch becomes too much large you can use a string mixin to shrink it to
few lines.

Bye,
bearophile
Jul 28 2010
parent reply Jason Spencer <spencer8 sbcglobal.net> writes:
== Quote from bearophile (bearophileHUGS lycos.com)'s article
Thanks for all the suggestions!

A little more discussion:


 So I need some slick way of mapping the run-time element size, row count,
 and column count into a static array type to instantiate my template with.





 A basic brutal way to do that is to use a switch, that switches according to
the


runtime value to use the correctly compile-time instantiated template. If the
switch
becomes too much large you can use a string mixin to shrink it to few lines.

I had thought of the switch, and it would be too large.  Can you give me an
idea of what
the mixin solution would look like?  Suppose I have three strings, type, rows,
and cols
that get set at runtime, where type = ["int" | "float" | "short"], rows and
cols =
["100" | "200" | "250"].  What I want is to take these 3 variables and come up
with:
foo!(int[100][100])(...)
foo!(int[100][200])(...)
...
foo!(short[250][250])(...)

for the right cases, without switching on type, rows, or cols.

Thanks again.
Jason
Jul 28 2010
parent reply bearophile <bearophileHUGS lycos.com> writes:
Jason Spencer:

 I had thought of the switch, and it would be too large.  Can you give me an
idea of what
 the mixin solution would look like?  Suppose I have three strings, type, rows,
and cols
 that get set at runtime, where type = ["int" | "float" | "short"], rows and
cols =
 ["100" | "200" | "250"].  What I want is to take these 3 variables and come up
with:
 foo!(int[100][100])(...)
 foo!(int[100][200])(...)
 ...
 foo!(short[250][250])(...)
 
 for the right cases, without switching on type, rows, or cols.


The brutal way:


import std.stdio: writeln;

void foo(T, int N, int M)() {
    writeln(typeid(T), " ", N, " ", M);
}

enum Type { int_type, float_type, short_type }
enum Size { small = 100, medium = 200, large = 250 }


void main() {
    Type type = Type.int_type;
    Size n = Size.medium;
    Size m = Size.small;


    final switch (type) {
        case Type.int_type:
            final switch (n) {
                case Size.small:
                    final switch (m) {
                        case Size.small:  foo!(int, Size.small, Size.small)();
break;
                        case Size.medium: foo!(int, Size.small, Size.medium)();
break;
                        case Size.large:  foo!(int, Size.small, Size.large)();
break;
                    }
                    break;
                case Size.medium:
                    final switch (m) {
                        case Size.small:  foo!(int, Size.medium, Size.small)();
break;
                        case Size.medium: foo!(int, Size.medium,
Size.medium)(); break;
                        case Size.large:  foo!(int, Size.medium, Size.large)();
break;
                    }
                    break;
                case Size.large:
                    final switch (m) {
                        case Size.small:  foo!(int, Size.large, Size.small)();
break;
                        case Size.medium: foo!(int, Size.large, Size.medium)();
break;
                        case Size.large:  foo!(int, Size.large, Size.large)();
break;
                    }
                    break;
            }
            break;

        case Type.float_type:
            final switch (n) {
                case Size.small:
                    final switch (m) {
                        case Size.small:  foo!(float, Size.small,
Size.small)(); break;
                        case Size.medium: foo!(float, Size.small,
Size.medium)(); break;
                        case Size.large:  foo!(float, Size.small,
Size.large)(); break;
                    }
                    break;
                case Size.medium:
                    final switch (m) {
                        case Size.small:  foo!(float, Size.medium,
Size.small)(); break;
                        case Size.medium: foo!(float, Size.medium,
Size.medium)(); break;
                        case Size.large:  foo!(float, Size.medium,
Size.large)(); break;
                    }
                    break;
                case Size.large:
                    final switch (m) {
                        case Size.small:  foo!(float, Size.large,
Size.small)(); break;
                        case Size.medium: foo!(float, Size.large,
Size.medium)(); break;
                        case Size.large:  foo!(float, Size.large,
Size.large)(); break;
                    }
                    break;
            }
            break;

        case Type.short_type:
            final switch (n) {
                case Size.small:
                    final switch (m) {
                        case Size.small:  foo!(short, Size.small,
Size.small)(); break;
                        case Size.medium: foo!(short, Size.small,
Size.medium)(); break;
                        case Size.large:  foo!(short, Size.small,
Size.large)(); break;
                    }
                    break;
                case Size.medium:
                    final switch (m) {
                        case Size.small:  foo!(short, Size.medium,
Size.small)(); break;
                        case Size.medium: foo!(short, Size.medium,
Size.medium)(); break;
                        case Size.large:  foo!(short, Size.medium,
Size.large)(); break;
                    }
                    break;
                case Size.large:
                    final switch (m) {
                        case Size.small:  foo!(short, Size.large,
Size.small)(); break;
                        case Size.medium: foo!(short, Size.large,
Size.medium)(); break;
                        case Size.large:  foo!(short, Size.large,
Size.large)(); break;
                    }
                    break;
            }
            break;
    }
}



A more compressed way:

import std.metastrings: Format;
import std.stdio: writeln;

void foo(T, int N, int M)() {
    writeln(typeid(T), " ", N, " ", M);
}

enum Type { int_type, float_type, short_type }
enum Size { small = 100, medium = 200, large = 250 }

void main() {
    Type type = Type.short_type;
    Size n = Size.medium;
    Size m = Size.small;

    enum string inner = "
                        case Size.small:  foo!(%s, Size.%s, Size.small)();
break;
                        case Size.medium: foo!(%s, Size.%s, Size.medium)();
break;
                        case Size.large:  foo!(%s, Size.%s, Size.large)();
break;
    ";


    enum string medial = "
                    final switch (m) {
                        mixin(Format!(inner, ty, si, ty, si, ty, si));
                    }
                    break;
    ";

    enum string outer = `
            final switch (n) {
                case Size.small:
                    enum string si = "small";
                    mixin(medial);
                case Size.medium:
                    enum string si = "medium";
                    mixin(medial);
                case Size.large:
                    enum string si = "large";
                    mixin(medial);
            }
            break;
    `;

    final switch (type) {
        case Type.int_type:
            enum string ty = "int";
            mixin(outer);

        case Type.float_type:
            enum string ty = "float";
            mixin(outer);

        case Type.short_type:
            enum string ty = "short";
            mixin(outer);
    }
}


It's a mess still :-)

Bye,
bearophile
Jul 28 2010
parent reply bearophile <bearophileHUGS lycos.com> writes:
This is better, probably it can be improves a bit more:

import std.stdio: writeln;
import std.metastrings: Format;
import std.typetuple: TypeTuple;

template Iota(int stop) {
    static if (stop <= 0)
        alias TypeTuple!() Iota;
    else
        alias TypeTuple!(Iota!(stop-1), stop-1) Iota;
}

void foo(T, int N, int M)() {
    writeln(typeid(T), " ", N, " ", M);
}

enum string[] types = ["int", "float", "short"];
enum string[] sizes = ["small", "medium", "large"];

enum Type : string { int_type = types[0], float_type = types[1], short_type =
types[2] }
enum Size { small = 100, medium = 200, large = 250 }

void main() {
    Type run_t = Type.int_type;
    Size run_n = Size.medium;
    Size run_m = Size.small;

    enum string pattern = "if (run_t == Type.%s_type && run_n == Size.%s &&
run_m == Size.%s) { foo!(%s, Size.%s, Size.%s)(); goto END; }";

    foreach (t; Iota!(types.length))
        foreach (n; Iota!(sizes.length))
            foreach (m; Iota!(sizes.length))
                mixin(Format!(pattern, types[t], sizes[n], sizes[m], types[t],
sizes[n], sizes[m]));

    END: {}
}


Bye,
bearophile
Jul 28 2010
parent reply bearophile <bearophileHUGS lycos.com> writes:
Simpler:

import std.stdio: writeln;
import std.typetuple: TypeTuple;

template Iota(int stop) {
    static if (stop <= 0)
        alias TypeTuple!() Iota;
    else
        alias TypeTuple!(Iota!(stop-1), stop-1) Iota;
}

void foo(T, int N, int M)() {
    writeln(typeid(T), " ", N, " ", M);
}

enum string[] types = ["int", "float", "short"];
alias TypeTuple!(int, float, short) types2;

enum string[] sizes = ["100", "200", "250"];
enum int[] sizes2 = [100, 200, 250];

void main() {
    string run_t = "int";
    string run_n = "200";
    string run_m = "100";

    foreach (t; Iota!(types.length))
        foreach (n; Iota!(sizes.length))
            foreach (m; Iota!(sizes.length))
                if (run_t == types[t] && run_n == sizes[n] && run_m ==
sizes[m]) {
                    foo!(types2[t], sizes2[n], sizes2[m])();
                    goto END;
                }

    END: {}
}
Jul 28 2010
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Last version for now, it can be improved further in various ways:


import std.typetuple: TypeTuple;
import std.metastrings: Format;

template Iota(int stop) {
    static if (stop <= 0)
        alias TypeTuple!() Iota;
    else
        alias TypeTuple!(Iota!(stop-1), stop-1) Iota;
}

enum string[] str_types = ["int", "float", "short"];
mixin(Format!("alias TypeTuple!(%s, %s, %s) types;", str_types[0],
str_types[1], str_types[2]));
static assert(str_types.length == types.length); // safety

enum string[] str_sizes = ["100", "200", "250"];
mixin(Format!("enum int[] sizes = [%s, %s, %s];", str_sizes[0], str_sizes[1],
str_sizes[2]));
static assert(str_sizes.length == sizes.length); // safety

void staticDispatch3(alias templ)(string run_t, string run_n, string run_m) {
    foreach (t; Iota!(str_types.length))
        foreach (n; Iota!(str_sizes.length))
            foreach (m; Iota!(str_sizes.length))
                if (run_t == str_types[t] && run_n == str_sizes[n] && run_m ==
str_sizes[m]) {
                    templ!(types[t], sizes[n], sizes[m])();
                    goto END;
                }
    assert(0); // safety

    END: {}
}

// ----------------------------

import std.stdio: writeln;

void foo(T, int N, int M)() {
    writeln(typeid(T), " ", N, " ", M);
}

void main() {
    staticDispatch3!(foo)("int", "200", "100");
}


A more generic staticDispatch3 like this doesn't work because items1 is a
tuple, so it gets flattened...

void staticDispatch3(alias func, alias str_items1, alias items1,
                                 alias str_items2, alias items2,
                                 alias str_items3, alias items3)
                    (string run_i1, string run_i2, string run_i3) {
    static assert(str_items1.length == items1.length);
    static assert(str_items2.length == items2.length);
    static assert(str_items3.length == items3.length);

    foreach (i1; Iota!(items1.length))
        foreach (i2; Iota!(items2.length))
            foreach (i3; Iota!(items3.length))
                if (run_i1 == str_types[i1] &&
                    run_i2 == str_types[i2] &&
                    run_i3 == str_types[i3]) {
                    func!(items1[i1], items2[i2], items3[i3])();
                    goto END;
                }
    assert(0); // safety

    END: {}
}

Bye,
bearophile
Jul 28 2010
parent reply Jason Spencer <spencer8 sbcglobal.net> writes:
Ok, I've gone over this, adapted it, and mostly understand it.  I just
have one question left:

== Quote from bearophile (bearophileHUGS lycos.com)'s article

 template Iota(int stop) {
 ...
         alias TypeTuple!(Iota!(stop-1), stop-1) Iota;
 }
 ...
     foreach (t; Iota!(str_types.length))


What happens at compile-time with this foreach loop?

I nievely went and replaced "foreach (t; Iota!(str_types.length))"
with "foreach (t; str_types.length)", since the length of that array
is known at compile-time.  That of course bombed, but I don't quite
get why.  Is the compiler actually evaluating the foreach loop at
compile time?  How could it, when the body makes run-time checks?  If
it's not, why doesn't my change work?

Jason
Jul 29 2010
next sibling parent reply Rory Mcguire <rjmcguire gm_no_ail.com> writes:
Jason Spencer wrote:


 Ok, I've gone over this, adapted it, and mostly understand it.  I just
 have one question left:
 
 == Quote from bearophile (bearophileHUGS lycos.com)'s article

 template Iota(int stop) {
 ...
         alias TypeTuple!(Iota!(stop-1), stop-1) Iota;
 }
 ...
     foreach (t; Iota!(str_types.length))


 
 What happens at compile-time with this foreach loop?
 
 I nievely went and replaced "foreach (t; Iota!(str_types.length))"
 with "foreach (t; str_types.length)", since the length of that array
 is known at compile-time.  That of course bombed, but I don't quite
 get why.  Is the compiler actually evaluating the foreach loop at
 compile time?  How could it, when the body makes run-time checks?  If
 it's not, why doesn't my change work?
 
 Jason


your replacement tries to loop over an uint called str_types.length. Never 
gonna happen.

Iota!(str_types.length) seems to generate str_types.length(a number of) 
integer indexes. Can't use 0 .. str_types.length in the foreach because 
compiler is expecting Integer constants so it can make the template "foo" 
into actual code.
Jul 29 2010
parent reply Jason Spencer <spencer8 sbcglobal.net> writes:
== Quote from Rory Mcguire (rjmcguire gm_no_ail.com)'s article

 Jason Spencer wrote:



 I nievely went and replaced "foreach (t; Iota!(str_types.length))"
 with "foreach (t; str_types.length)", since the length of that
 array is known at compile-time.





 your replacement tries to loop over an uint called str_types.length.
 Never gonna happen.
 Iota!(str_types.length) seems to generate str_types.length(a number
 of)integer indexes. Can't use 0 .. str_types.length in the foreach
 because compiler is expecting Integer constants so it can make the
 template "foo" into actual code.


Not quite sure I follow.  I think you're saying the range in foreach
has to be actual literals, and not just an expression that can be
evaluated at compile time to generate the same range...close?

If that's the case, then why does it work to instantiate Iota! with
str_types.length?  It can obviously get the value behind it at compile
time.  I'm still missing something.

Jason
Jul 30 2010
parent reply Rory Mcguire <rjmcguire gm_no_ail.com> writes:
Jason Spencer wrote:


 == Quote from Rory Mcguire (rjmcguire gm_no_ail.com)'s article

 Jason Spencer wrote:


 

 I nievely went and replaced "foreach (t; Iota!(str_types.length))"
 with "foreach (t; str_types.length)", since the length of that
 array is known at compile-time.




 

 your replacement tries to loop over an uint called str_types.length.
 Never gonna happen.
 Iota!(str_types.length) seems to generate str_types.length(a number
 of)integer indexes. Can't use 0 .. str_types.length in the foreach
 because compiler is expecting Integer constants so it can make the
 template "foo" into actual code.


 
 Not quite sure I follow.  I think you're saying the range in foreach
 has to be actual literals, and not just an expression that can be
 evaluated at compile time to generate the same range...close?
 
 If that's the case, then why does it work to instantiate Iota! with
 str_types.length?  It can obviously get the value behind it at compile
 time.  I'm still missing something.
 
 Jason


I convert str_types.length to its actual value below:
foreach (t; 3) {
...
}

You can't do that (dmd : t.d(6): Error: int is not an aggregate type)
Jul 30 2010
parent reply Jason Spencer <spencer8 sbcglobal.net> writes:
== Quote from Rory Mcguire (rjmcguire gm_no_ail.com)'s article

 Jason Spencer wrote:

 == Quote from Rory Mcguire (rjmcguire gm_no_ail.com)'s article

 Jason Spencer wrote:



 I nievely went and replaced "foreach (t;








Iota!(str_types.length))"

 with "foreach (t; str_types.length)", since the length of that
 array is known at compile-time.









 Can't use 0 .. str_types.length in the foreach
 because compiler is expecting Integer constants so it can make
 the template "foo" into actual code.






This is the part I'm still not getting.  Why shouldn't

   foreach (t; 0..3)

work?  Those are integer constants.

Actually, I think I'm getting it.  str_types.length is actually (or
close to) an integer literal, but t is not.  t over a range is an int
variable.  So at best, the compiler will infer the type of t and try
to get TypeTuple![int] from the mixin, which doesn't help.  But it
works in Iota because it only needs a value, and the length property
is not a variable, but a compile-time constant.

I'm not sure what magic gets worked when t is bound to a TypeTuple
that has int literals, but I'm guessing t in that case is not an int
variable, but a compile-time type variable, and it iterates over int
literals.  Those work with templ.  What I really want to know is "does
that foreach run at compile-time or run-time?"  I suspect compile-time
because it iterates over type variables.  But documentation is shakey
:)




 I convert str_types.length to its actual value below:
 foreach (t; 3) {
 ...
 }
 You can't do that (dmd : t.d(6): Error: int is not an aggregate


type)

Yeah, I mis-typed orginally.  It was "foreach (t;
0..str_types.length)" as a range.
Jul 31 2010
parent Rory Mcguire <rjmcguire gm_no_ail.com> writes:
Jason Spencer wrote:


 == Quote from Rory Mcguire (rjmcguire gm_no_ail.com)'s article

 Jason Spencer wrote:

 == Quote from Rory Mcguire (rjmcguire gm_no_ail.com)'s article

 Jason Spencer wrote:



 I nievely went and replaced "foreach (t;








 Iota!(str_types.length))"

 with "foreach (t; str_types.length)", since the length of that
 array is known at compile-time.








 

 Can't use 0 .. str_types.length in the foreach
 because compiler is expecting Integer constants so it can make
 the template "foo" into actual code.






 
 This is the part I'm still not getting.  Why shouldn't
 
    foreach (t; 0..3)
 
 work?  Those are integer constants.
 
 Actually, I think I'm getting it.  str_types.length is actually (or
 close to) an integer literal, but t is not.  t over a range is an int
 variable.  So at best, the compiler will infer the type of t and try
 to get TypeTuple![int] from the mixin, which doesn't help.  But it
 works in Iota because it only needs a value, and the length property
 is not a variable, but a compile-time constant.
 
 I'm not sure what magic gets worked when t is bound to a TypeTuple
 that has int literals, but I'm guessing t in that case is not an int
 variable, but a compile-time type variable, and it iterates over int
 literals.  Those work with templ.  What I really want to know is "does
 that foreach run at compile-time or run-time?"  I suspect compile-time
 because it iterates over type variables.  But documentation is shakey
 :)
 
 
 

 I convert str_types.length to its actual value below:
 foreach (t; 3) {
 ...
 }
 You can't do that (dmd : t.d(6): Error: int is not an aggregate


 type)
 
 Yeah, I mis-typed orginally.  It was "foreach (t;
 0..str_types.length)" as a range.


The foreach using Iota is unrolled at compile time, bearofile mentioned this 
somewhere as well.
Aug 03 2010
prev sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Jason Spencer:

 I nievely went and replaced "foreach (t; Iota!(str_types.length))"
 with "foreach (t; str_types.length)", since the length of that array
 is known at compile-time.  That of course bombed, but I don't quite
 get why.  Is the compiler actually evaluating the foreach loop at
 compile time?  How could it, when the body makes run-time checks?  If
 it's not, why doesn't my change work?


foreach (t; str_types.length) doesn't work because integral values are not
iterable. You need a range, collection or tuple.

foreach (t; 0 .. str_types.length) doesn't work in that code because it's a
run-time foreach.

The  Iota!(n) creates a tuple of the values 0, 1, 2 ... n-1.

The foreach done on a tuple is a static foreach, it's done at compile-time, so
my code is a way to perform a compile-time unrolling of the if-goto body. You
can see it with this example (normally DMD is not able to perform loop
unrolling):


import std.typetuple: TypeTuple;
import std.c.stdio: printf;

template Iota(int stop) {
    static if (stop <= 0)
        alias TypeTuple!() Iota;
    else
        alias TypeTuple!(Iota!(stop-1), stop-1) Iota;
}
void main() {
    foreach (i; Iota!(5)) {
        enum int n = i;
        printf("%d\n", n);
    }
}


The resulting asm:

__Dmain comdat
        push    EBX
        push    0
        mov EAX,offset FLAT:_DATA
        push    EAX
        call    near ptr _printf
        add ESP,8
        push    1
        mov ECX,offset FLAT:_DATA
        push    ECX
        call    near ptr _printf
        add ESP,8
        push    2
        mov EDX,offset FLAT:_DATA
        push    EDX
        call    near ptr _printf
        add ESP,8
        push    3
        mov EBX,offset FLAT:_DATA
        push    EBX
        call    near ptr _printf
        add ESP,8
        push    4
        push    EBX
        call    near ptr _printf
        add ESP,8
        xor EAX,EAX
        pop EBX
        ret


I suggest you to use a variant of my last version of the code because it
contains some asserts and static asserts that improve code safety a bit.

See also this enhancement request of mine:
http://d.puremagic.com/issues/show_bug.cgi?id=4085

Bye,
bearophile
Jul 30 2010
parent Jason Spencer <spencer8 sbcglobal.net> writes:
 bearophile:

Got it.  Had it mostly worked out before reading, but this helps clarify
the static foreach part.

Thanks, gang!

Jason
Jul 31 2010
prev sibling parent Jason Spencer <spencer8 sbcglobal.net> writes:
That's COOL!

I'll have to look at these closer, but I definitely get what you're
doing.  Thanks a million.

== Quote from bearophile (bearophileHUGS lycos.com)'s article

 Simpler:
 import std.stdio: writeln;
 import std.typetuple: TypeTuple;
 template Iota(int stop) {
     static if (stop <= 0)
         alias TypeTuple!() Iota;
     else
         alias TypeTuple!(Iota!(stop-1), stop-1) Iota;
 }
 void foo(T, int N, int M)() {
     writeln(typeid(T), " ", N, " ", M);
 }
 enum string[] types = ["int", "float", "short"];
 alias TypeTuple!(int, float, short) types2;
 enum string[] sizes = ["100", "200", "250"];
 enum int[] sizes2 = [100, 200, 250];
 void main() {
     string run_t = "int";
     string run_n = "200";
     string run_m = "100";
     foreach (t; Iota!(types.length))
         foreach (n; Iota!(sizes.length))
             foreach (m; Iota!(sizes.length))
                 if (run_t == types[t] && run_n == sizes[n] && run_m


== sizes[m]) {

                     foo!(types2[t], sizes2[n], sizes2[m])();
                     goto END;
                 }
     END: {}
 }
Jul 28 2010
prev sibling next sibling parent reply Kagamin <spam here.lot> writes:
Jason Spencer Wrote:


 If I want to avoid the copy, am I relegated back
 to pointers?


you can make a thin wrapper that will work as a reference-type static array

struct ReferenceArray!(ElementType, int columns, int rows)
{
  ElementType[columns][rows]* back;
  this(byte[] data)
  {
    assert(data.length==ElementType.sizeof*columns*rows);
    back = cast(ElementType[columns][rows]*)data.ptr;
  }
  //will it be inlined?
  ElementType opIndex(int column, int row)
  {
    return (*back)[row][column];
  }
}
Jul 29 2010
parent Kagamin <spam here.lot> writes:
Kagamin Wrote:


 Jason Spencer Wrote:
 

 If I want to avoid the copy, am I relegated back
 to pointers?


 


or something like this

struct ReferenceArray!(ArrayType)
{
  ArrayType* back;
  this(byte[] data)
  {
    assert(data.length==ArrayType.sizeof);
    back = cast(ArrayType*)data.ptr;
  }
  //will it be inlined?
  auto opIndex(int column, int row)
  {
    return (*back)[row][column];
  }
}

byte[] data = getData();
ReferenceArray!(float[100][100]) matrix = data;
writeln(matrix[1][2]);
Jul 29 2010
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 28 Jul 2010 14:52:11 -0400, Jason Spencer <spencer8 sbcglobal.net>  
wrote:


 I'm working on a program to do statistics on matrices of different  
 sizes, and
 I've run into a handful of situations where I just can't seem to find the
 trick I need.  In general, I'm trying to make my functions work on static
 arrays of the proper size, and template that up for the different sizes  
 I need
 to support.  I appeal to your experience and mercy on the various  
 questions
 below.  Any help on any point appreciated.

 1.  How can I cast a single dimension dynamic array to a multi-dimension
 static array?  I'm trying to do roughly the following:

    auto data = cast(float[100][100])std.file.read(fname, fsize);

 which fails on the cast.  Is there any way to treat they memory returned
 from read as a static array?  If I want to avoid the copy, am I  
 relegated back
 to pointers?  Is there a cast path through a pointer that would work?  I  
 think
 I'm essentially asking if I can make a reference to a static array and  
 assign
 that to the memory contained in a dynamic array.  I have a suspicion  
 about the
 answer....

 2.  To work around 1., I've left one dimension of the cast floating.   
 This
 means to bind both dimensions in my template args, I can no longer use
 type parameter inference, so I have to explicitly instantiate the  
 template
 with the static size, but pass in a type that's dynamic in one dimension.


That is the correct way to do this.  std.file.read is reading data into a  
heap array.  If you want to cast this into a static array that lives on  
the stack, you will end up copying the data from the heap to the stack.   
Essentially, you are asking the type system to ignore the fact that  
std.file.read may *not* read 100x100 floats, which is quite unsafe.

Another thing you can do:

float[100][100] data;
File f = File(fname);
enforce(std.file.rawRead(data[]).length == data.length);

Now, data should be filled in, and you haven't used the heap.


 If I could solve that, then next I'd need to make this work:

    U foo(T: U[C][R], U, size_t C, size_t R)(U[C][] data, size_t c,  
 size_t r)
    {
       return data[r][c];
    }

    int[4][] data = [ [...] ];
    int bar = foo(data, 2, 3);

 Extra points for that one!


That one will never work, because data's type does not include the main  
dimension, so it can't be decided at compile time.

-Steve
Jul 29 2010