www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Creating Structs/Classes at runtime?

reply "Brian Brady" <brian.brady1982 gmail.com> writes:
Hi

Ok first ... is this possible? Or wise?
Being mostly self taught with regards to programming I sometimes 
wonder if the way I'm going about something is even the right 
approach, never mind whether my code is logical. So if this seems 
ludicrous then please tell me my approach is silly and point me 
in the right direction ... :)

Ok so ...

I was reading in some data from a csv and wanted to read it into 
a struct called.
My problem is
1) my full csv is 4000 variables wide, and I really don't want to 
declare each variable in the struct (I've created a smaller one 
to test)
2) what if I wanted to read in different csvs of different sizes, 
I don't want to have to change my program each time?

After much googling, I thought I found something on RosettaCode 
that would do what I wanted 
(http://rosettacode.org/wiki/Add_a_variable_to_a_class_instance_at_runtime#D)
but when I try to use it, it doesn't work. (Its actually adding 
to a class, which may be better than a struct in this instance, 
but regardless, it doesn't work)

As my programming skills are mainly obtained through finding 
other work that does something similar to what I desire and 
stealing/hacking it, please be gentle if there is some glaringly 
foolish mistake that is obvious. :S

Thanks in advance
Brian

My code is:
module main;

import std.stdio;
import std.csv;
import std.string;
import std.file;
import std.array;
import std.variant;

class dataVector(T)
{
     private T[string] vars;
      property T opDispatch(string key)() pure nothrow
     {
         return vars[key];
     }

      property void opDispatch(string key, U)(U value)/*pure*/ 
nothrow
     {
         vars[key] = value;
     }
};

void fillVector(int size, string filenameToUse)
{
     writeln(size);
     uint loopNum = size;
     do
     {
         writeln(loopNum);
     //    auto test = dataVector!Variant();
/*
         auto file = File(filenameToUse, "r");
         foreach(line;file.byLine())
         {
             foreach(ob;csvReader!Data(text))
             {
                 dataVector.size ~= ob;
             }
         }*/
// ****************************************
// according to the RosettaCode this is how I should be able
// to create a variable a in the class test with a value of 
loopNum + 1
// ****************************************
         test.a = loopNum + 1;

  //       writeln(test.a);
         loopNum--;
     }
     while(loopNum != 0);
}

void main()
{
     string input;

     string filename = "/A/Path/To/The/Dark/Side/output.csv";
     auto file = File(filename, "r");

/* this is super dynamic, but I've managed to create a different 
function to calculate this, which I have omitted to keep the code 
concise */
     int numOfVars = 11;
     if (numOfVars > 0)
     fillVector(numOfVars, filename);
}

The csv I would eventually like to read in is of the form:

test1,303,-140,-166,-317,-310,414,-157,-360,-120,-89
test10,-1,70,-57,-101,112,137,-134,9,195,86
test100,367,78,-417,123,220,-234,-170,236,-218,-351
test1000,309,-178,-674,-202,514,218,-165,76,-82,-328
test10000,-131,142,6,-143,80,46,29,48,-84,-113
Feb 17 2013
next sibling parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
There is a way to get what you described, and your code is fairly 
close... but there's a much simpler way too.

from the std.csv docs:

==

  When an input contains a header the $(D Contents) can be 
specified as an
  associative array. Passing null to signify that a header is 
present.

  -------
  auto text = "Name,Occupation,Salary\r"
      "Joe,Carpenter,300000\nFred,Blacksmith,400000\r\n";

  foreach(record; csvReader!(string[string])
          (text, null))
  {
      writefln("%s works as a %s and earns $%s per year.",
               record["Name"], record["Occupation"],
               record["Salary"]);
  }
  -------

==

The associative array directly is the simplest way to get this to 
work. All items read will be of type string if you do it like 
this, and you access with the [] syntax.

You can convert to other types with to when you want to use it:

import std.conv;

int salary = to!int(record["salary"]);



and that's the simplest way to make it work.
Feb 17 2013
parent "Brian Brady" <brian.brady1982 gmail.com> writes:
On Monday, 18 February 2013 at 00:52:12 UTC, Adam D. Ruppe wrote:
[...]

  -------
  auto text = "Name,Occupation,Salary\r"
      "Joe,Carpenter,300000\nFred,Blacksmith,400000\r\n";

  foreach(record; csvReader!(string[string])
          (text, null))
  {
      writefln("%s works as a %s and earns $%s per year.",
               record["Name"], record["Occupation"],
               record["Salary"]);
  }


doesn't this writefln() assume that I know what the Variables are 
called? ie "Name", "Occupation", etc.?
I want to be able to run the same program against a file with 4 
variables or a file with 400 variables, so specifying names 
wouldn't work.
Can I somehow use a record[var[a]] where a can be a number 
between 0 and the count of variables?

Also, if I wanted to store this as I read it in from csvReader, 
how can I define the array? It'll be of differing sizes if my 2 
csv's are of different sizes. This kinda the crux of my problem.

string[string] Data;
is my first guess based on http://dlang.org/hash-map.html

but then this doesn't work:
      foreach(line;file.byLine())
          foreach(ob;csvReader!Data(line))
          {
              // do things
          }
Feb 17 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Feb 18, 2013 at 01:44:49AM +0100, Brian Brady wrote:
[...]

 I was reading in some data from a csv and wanted to read it into a
 struct called.
 My problem is
 1) my full csv is 4000 variables wide, and I really don't want to
 declare each variable in the struct (I've created a smaller one to
 test)
 2) what if I wanted to read in different csvs of different sizes, I
 don't want to have to change my program each time?


[...]

It seems to me that what you need is not any struct or class per se, but
a runtime container that stores different kinds of data depending on
runtime input. I would recommend using an associative array instead of
trying to shoehorn dynamic data into struct fields. Say something like:

	import std.variant;

	Variant[string] data;
	data[fieldName] = value;
	...
	auto myField = data[fieldName];
	...

Etc.


T

-- 
There is no gravity. The earth sucks.
Feb 17 2013
prev sibling next sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 02/17/2013 04:44 PM, Brian Brady wrote:


 1) my full csv is 4000 variables wide, and I really don't want to
 declare each variable in the struct (I've created a smaller one to test)


Looking at the sample file you provide, what you call "variables" look 
like data points.


 2) what if I wanted to read in different csvs of different sizes, I
 don't want to have to change my program each time?


Then you need something other than a CSV reader. Ordinarily, records in 
a CSV files have a known number of fields.


 After much googling, I thought I found something on RosettaCode that
 would do what I wanted
 


(http://rosettacode.org/wiki/Add_a_variable_to_a_class_instance_at_runtime#D) 


 but when I try to use it, it doesn't work. (Its actually adding to a
 class, which may be better than a struct in this instance, but
 regardless, it doesn't work)


That is not a natural idiom for D and I don't think it is needed here. :)


 The csv I would eventually like to read in is of the form:

 test1,303,-140,-166,-317,-310,414,-157,-360,-120,-89
 test10,-1,70,-57,-101,112,137,-134,9,195,86
 test100,367,78,-417,123,220,-234,-170,236,-218,-351
 test1000,309,-178,-674,-202,514,218,-165,76,-82,-328
 test10000,-131,142,6,-143,80,46,29,48,-84,-113


What I see there is a label and a number of integers. Here is a simple 
parser that takes advantage of the %( and %) grouping format specifiers:

import std.stdio;
import std.format;

struct Data
{
     string label;
     int[] values;
}

int main(string[] args)
{
     if (args.length != 2) {
         stderr.writefln("Usage: %s <input-file-name>", args[0]);
         return 1;
     }

     auto file = File(args[1], "r");

     Data[] data;

     foreach (line; file.byLine) {
         string label;
         int[] values;
         formattedRead(line, "%s,%(%s,%)", &label, &values);
         data ~= Data(label, values);
     }

     writeln(data);

     return 0;
}

Ali

-- 
D Programming Language Tutorial: http://ddili.org/ders/d.en/index.html
Feb 17 2013
parent reply "Brian Brady" <brian.brady1982 gmail.com> writes:
On Monday, 18 February 2013 at 05:26:39 UTC, Ali Çehreli wrote:

Thank you for the working solution.

[...]

 Looking at the sample file you provide, what you call 
 "variables" look like data points.


Yes, apologies. Different languages using different terms to 
describe, essentially the same thing.

[...]

 Then you need something other than a CSV reader. Ordinarily, 
 records in a CSV files have a known number of fields.


While you would assume the lines within a csv contain a specific 
number of 'data points' surely the width of the csv is as 
variable as the data within them? This is what I want to code 
around.

[...]

 That is not a natural idiom for D and I don't think it is 
 needed here. :)


That's ok. Like I said, I open to be pointed in a new, better 
direction.

[...]

 What I see there is a label and a number of integers. Here is a 
 simple parser that takes advantage of the %( and %) grouping 
 format specifiers:


Aye, in this instance it is just that, but I was looking for a 
more generalised code to account for, for example, when the first 
2 are labels, or the first and third ...

Thanks everyone for the help.

Regards
Brian
Feb 18 2013
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 02/18/2013 02:55 AM, Brian Brady wrote:

 On Monday, 18 February 2013 at 05:26:39 UTC, Ali Çehreli wrote:



 What I see there is a label and a number of integers. Here is a simple
 parser that takes advantage of the %( and %) grouping format specifiers:


 Aye, in this instance it is just that, but I was looking for a more
 generalised code to account for, for example, when the first 2 are
 labels, or the first and third ...


You mean like this?

10,20,label,1,2,3,4

Then what are 10 and 20 on that line? Do they belong to the previous 
label? If so, I think this format is too free-form to be parsed by a 
general solution like csvReader. It looks like something special needs 
to be done and it is hard to say without knowing the meanings of 10 and 
20 above. :)

Ali
Feb 18 2013
parent reply "Brian Brady" <brian.brady1982 gmail.com> writes:
On Monday, 18 February 2013 at 16:54:59 UTC, Ali Çehreli wrote:
[...]

 You mean like this?

 10,20,label,1,2,3,4

 Then what are 10 and 20 on that line? Do they belong to the 
 previous label? If so, I think this format is too free-form to 
 be parsed by a general solution like csvReader. It looks like 
 something special needs to be done and it is hard to say 
 without knowing the meanings of 10 and 20 above. :)

 Ali


I don't mind reading them all in as strings, and then converting 
them. I imagine I can do that relatively easily (famous last 
words) by checking if all the entries in the string are numeric 
and if so casting to an int/long/double etc.

My question was aimed at how to store them really. What was the 
best dynamic container type that I could use to store data points 
read from csvs.

I want something which is robust enough that I don't have to keep 
changing the code every time I change my csv.

So a program that reads in a csv of the form:

"string, number, number, number"

and stores it in a container/matrix/associative array

could also read in

"number, number, string, number, string, number, number, number"

and store it in a container with similar properties.
Once I can get both csvs input in the same manner, I can work on 
determining what is in each container, and go from there.

While this may seem madness, I hope my explanation describes what 
I am trying to achieve?

Brian
Feb 18 2013
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 02/18/2013 04:31 PM, Brian Brady wrote:


 So a program that reads in a csv of the form:

 "string, number, number, number"

 and stores it in a container/matrix/associative array

 could also read in

 "number, number, string, number, string, number, number, number"

 and store it in a container with similar properties.
 Once I can get both csvs input in the same manner, I can work on
 determining what is in each container, and go from there.


How about an OO solution?

import std.stdio;

interface Data
{
     void doSomething();
}

// This works with any simple type
class SimpleData(T) : Data
{
     T value;

     this(T value)
     {
         this.value = value;
     }

     void doSomething()
     {
         writefln("Doing something with %s %s", T.stringof, value);
     }
}

alias IntData = SimpleData!int;
alias StringData = SimpleData!string;

// My special type
struct MyStruct
{
     double d;
     string s;

     void foo()
     {
         writefln("Inside MyStruct.foo");
     }
}

// This works with my special type
class MyStructData : Data
{
     MyStruct value;

     this(double d, string s)
     {
         this.value = MyStruct(d, s);
     }

     void doSomething()
     {
         writefln("Doing something special with this MyStruct: %s", value);
         value.foo();
     }
}

void main()
{
     Data[] dataContainer;

     // Append according to what gets parsed from the input
     dataContainer ~= new IntData(42);
     dataContainer ~= new StringData("hello");
     dataContainer ~= new MyStructData(1.5, "goodbye");

     // Use the data according to the Data interface
     foreach (data; dataContainer) {
         data.doSomething();
     }
}

The output:

Doing something with int 42
Doing something with string hello
Doing something special with this MyStruct: MyStruct(1.5, "goodbye")
Inside MyStruct.foo

Ali
Feb 18 2013
parent "Brian Brady" <brian.brady1982 gmail.com> writes:
On Tuesday, 19 February 2013 at 01:09:06 UTC, Ali Çehreli wrote:
[...]


 How about an OO solution?


That looks awesome. :)
I'll have a play with it and see if I can bend it to my will.

Thanks for the help.
Like I said, I'm a bit of a noob, so a push in a more suitable 
direction is always appreciated. :)

Regards
Brian
Feb 18 2013
prev sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Brian Brady:


 After much googling, I thought I found something on RosettaCode 
 that would do what I wanted 
 (http://rosettacode.org/wiki/Add_a_variable_to_a_class_instance_at_runtime#D)
 but when I try to use it, it doesn't work. (Its actually adding 
 to a class, which may be better than a struct in this instance, 
 but regardless, it doesn't work)


Since some time I am maintaining most of the D code on 
Rosettacode. What's broken in that program? "it doesn't work" is 
too much vague.

Bye,
bearophile
Feb 18 2013
parent "Brian Brady" <brian.brady1982 gmail.com> writes:
On Monday, 18 February 2013 at 12:28:22 UTC, bearophile wrote:
[...]

 Since some time I am maintaining most of the D code on 
 Rosettacode. What's broken in that program? "it doesn't work" 
 is too much vague.

 Bye,
 bearophile


Apologies. My wording was poor. I believe the RosettaCode code 
worked for that example, but when I tried to get it to work for 
my purposes I could not, and could not figure out why. Possibly 
because I was also trying to incorporate reading from a file at 
the same time.

To be honest, I'm not really sure what is going on in the struct 
definition to even know where my code was possibly failing. I'd 
like to take time to figure it out, but it has a lot of parts 
that I don't understand, so it'll be a while.

Cheers
Brian
Feb 18 2013