www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - String[] pointer to void* and back

reply "seany" <seany uni-bonn.de> writes:
Consider this snippet:



import std.stdio;
import std.conv;
import core.vararg;





void main() {

      string[] s = ["aa", "bb", "cc"];
      string []* ss;
      void * v;

      ss = &s;
      v = cast(void*)s;

      ss = cast(string[]*) v;

      s = *ss;

      writeln(s);

}

This fails, Stack overflow.

If s was a double array, it works.

What am I doing wrong?
Sep 18 2014
next sibling parent "seany" <seany uni-bonn.de> writes:
Found, it should have been       v = cast(void*)ss;

sorry.
Sep 18 2014
prev sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 09/18/2014 01:59 PM, seany wrote:


       string[] s = ["aa", "bb", "cc"];
       string []* ss;
       void * v;

       ss = &s;
       v = cast(void*)s;


Not s, but its address should be assigned to v:

     v = cast(void*)&s;

Only then it will match its reverse operation:


       ss = cast(string[]*) v;

       s = *ss;

       writeln(s);


Ali
Sep 18 2014
parent reply "seany" <seany uni-bonn.de> writes:
Yes, thank you, I corrected that.

However, if this v is a member of a class, like



import std.stdio;
import std.conv;
import core.vararg;

struct S
   {
     void *v;
   }

class C
{


   S* sx = new S;

   void dothings()
   {
      string[] ss = ["1", "2", "4"];
      string[] *s;
      void *vv;

      s = &ss;
      vv = s;

      sx.v = vv;

   }

}


void main() {

      C c = new C;
      c.dothings();
      writeln("done");

      string[]* sh;
      sh = cast(string[]*)c.sx.v;
      writeln(sh);        // upto this, works, the same pointer as 
set in
                          // c.dothings(), checked with write 
instructions

      string[] si = *sh;
      writeln(si);
}



and then casted back, then i notice that it does not work. 
Wondering why.
Sep 18 2014
next sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 09/18/2014 02:35 PM, seany wrote:


 struct S
    {
      void *v;
    }

 class C
 {


    S* sx = new S;

    void dothings()
    {
       string[] ss = ["1", "2", "4"];


Note that ss is a local variable of a druntime type equivalent of the 
following:

struct D_Slice_of_strings_
{
     size_t length;
     string * ptr;
}

Although the elements that are accessed through .ptr are in dynamic 
memory and owned by the GC, the local struct object (i.e. ss) itself is 
on the program stack. It will be gone upon leaving dothings().


       string[] *s;
       void *vv;

       s = &ss;
       vv = s;

       sx.v = vv;

    }


As a demonstration, one solution is to make ss a member variable. Then, 
it would live as long as v lived. However, ss can be in some other 
long-lived container as well.

Ali
Sep 18 2014
parent reply "seany" <seany uni-bonn.de> writes:
what if i needed to access many such runtime variables  of many 
types, and did not want to create a member for each type?
Sep 18 2014
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 09/18/2014 02:52 PM, seany wrote:

 what if i needed to access many such runtime variables  of many types,
 and did not want to create a member for each type?


If you are holding an address in a void*, you must make sure that the 
original object is still at that location when you attempt to access the 
object.

If there are limited number of such string[] arrays then you can 
populate an array in the beginning and pass around void* values to 
elements in there.

However, as soon as an element is added or removed from an array, all 
(or some) of the references to those elements get invalidated. (An 
exceptions is where the array has capacity and you add an element.)

Linked lists, trees, etc. don't have that problem: Their elements stay 
where they are even after elements are added to or removed from the 
container.

If you can, storing an index instead of a void* is a better way to go. 
Even if the elements are relocated, as long as no element is removed 
from the collection, an index value will always be valid.

Ali
Sep 18 2014
parent reply "seany" <seany uni-bonn.de> writes:
On Thursday, 18 September 2014 at 22:16:48 UTC, Ali Çehreli wrote:


 If you are holding an address in a void*, you must make sure 
 that the original object is still at that location when you 
 attempt to access the object.


Does that mean, that there is no way to make a global stack 
accessible accross modules, of runtime generated stack variables, 
unless i define such variable species beforehand?
Sep 19 2014
next sibling parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 09/19/2014 05:14 AM, seany wrote:


 On Thursday, 18 September 2014 at 22:16:48 UTC, Ali Çehreli wrote:


 If you are holding an address in a void*, you must make sure that the
 original object is still at that location when you attempt to access
 the object.


 Does that mean, that there is no way to make a global stack accessible
 accross modules, of runtime generated stack variables,


I assume you mean 'program stack' as opposed to the 'stack' data structure.

Objects on the program stack don't work because program stack is like a 
scratch pad and the stack data structure won't work in some situations 
if it's implemented in terms of an array and if the array gets larger 
over time, because array elements get moved around as the capacity is 
consumed.

The important part is to ensure that the object will be there when the 
void* is actually used.

Here is an example that uses a fixed-length array. string[] objects are 
passed as void* parameters to a function (perhaps a C library function). 
That function uses the void* argument when calling the callback function.

When the callback function received the void*, the object is still in 
the global (more correctly, module-scoped) array.

Note that although I used literals string arrays when populating the 
tables, they can be stack variables as well, as long as they are put in 
to the array and addresses of the array elements are used.

import std.stdio;

string[][2] stringTables;

void populateTables()
{
     stringTables[0] = [ "apple", "pear" ];
     stringTables[1] = [ "coffee", "tea" ];
}

void main()
{
     populateTables();

     foreach (i; 0 .. 4) {
         const index = i % 2;
         void* param = &(stringTables[index]);
         callWith(&callbackFunction, param);
     }
}

alias CallbackFunction = void function(void*);

void callWith(CallbackFunction func, void* param)
{
     func(param);
}

void callbackFunction(void* param)
{
     string[]* table = cast(string[]*)param;
     writefln("Called with %s", *table);
}


 unless i define such variable species beforehand?


Unlike the example above, you can define the objects during runtime as 
well. The problem with slices is that it is not possible to create the 
actual slice object dynamically:

     alias Numbers = int[];
     auto p = new Numbers;

Error: new can only create structs, dynamic arrays or class objects, not 
int[]'s

Ironically, the error message mentions 'dynamic arrays' but it is 
talking about the actual storage for elements, not the slice object itself.

So, one way is to wrap the slice in a struct:

struct Table
{
     int[] table;
}

void foo()
{
     void* p = new Table;
     // ...
}

Now it works and 'p' can be used as a void*. Although the struct object 
(and the slice inside it) is on the GC heap, there is a different kind 
of problem: There may not be any other pointer to that object and the GC 
may collect the memory before the void* is used in the program later on. 
(This may not be a concern if the void* remains in the program as a 
reference. However, if the void* is passed to e.g. a C library function, 
which puts it in a container there, then GC will not know about it.

Although GC.addRange helps with that, the void* can safely be put in a 
void* slice as well:

void*[] persistingTable;

     persistingTable ~= p;

This time, even if persistingTable moves its void* elements around, it 
won't matter because persistingTable is only for keeping a visible 
pointer in the program so that GC doesn't free our objects.

As you can see, there are a lot of things to consider. :) What are you 
thinking of using the void* for?

Ali
Sep 19 2014
prev sibling parent "anonymous" <anonymous example.com> writes:
On Friday, 19 September 2014 at 12:14:19 UTC, seany wrote:

 On Thursday, 18 September 2014 at 22:16:48 UTC, Ali Çehreli 
 wrote:


 If you are holding an address in a void*, you must make sure 
 that the original object is still at that location when you 
 attempt to access the object.


 Does that mean, that there is no way to make a global stack 
 accessible accross modules, of runtime generated stack 
 variables, unless i define such variable species beforehand?


I don't understand your usage of the word "stack". There may
confusion about the meaning of (the) stack.

As far as I understand, you want to store values (or references
to values) of arbitrary types. You tried to use `void*` for that.
That works, as long as the value stays where the pointer points.
Which is not the case when the value is on the call stack [1]
(i.e. "the stack"). If you put the value on "the heap" [2] (not
to be confused with the data structure), it works fine:

---
import std.stdio;

class C
{
    void* v;

    void dothings()
    {
       string[] ss = ["1", "2", "4"];
       string[][] onTheHeap = new string[][1]; /* Allocate space on
the heap. Can't `new` a single string[] directly, so make it an
array of length 1. */
       onTheHeap[0] = ss; /* Copy ss from the stack to the heap. */
       v = onTheHeap.ptr; /* Could also be `&onTheHeap[0]`, but not
`&onTheheap`. */
    }
}


void main()
{
     C c = new C;
     c.dothings();

     string[]* sh;
     sh = cast(string[]*)c.v;

     string[] si = *sh;
     writeln(si); /* ["1", "2", "4"] */
}
---

By the way, the struct S isn't really buying you anything over
using void* directly.

[1] http://en.wikipedia.org/wiki/Call_stack
[2] http://en.wikipedia.org/wiki/Heap_(programming)
Sep 19 2014
prev sibling parent reply "anonymous" <anonymous example.com> writes:
On Thursday, 18 September 2014 at 21:35:50 UTC, seany wrote:

 Yes, thank you, I corrected that.

 However, if this v is a member of a class, like



 import std.stdio;
 import std.conv;
 import core.vararg;

 struct S
   {
     void *v;
   }

 class C
 {


   S* sx = new S;

   void dothings()
   {
      string[] ss = ["1", "2", "4"];


ss is a local variable. I.e., ss.ptr and ss.length are on the
stack.


      string[] *s;
      void *vv;

      s = &ss;
      vv = s;


vv now holds a pointer to the stack.


      sx.v = vv;


Here, the pointer to the stack escapes the function. Don't do
that! When dothings returns, its portion of the stack becomes
unoccupied. Other functions will re-use it and stomp over the
data. The escaped pointer then points to some completely 
unrelated data.


   }

 }


 void main() {

      C c = new C;
      c.dothings();
      writeln("done");

      string[]* sh;
      sh = cast(string[]*)c.sx.v;
      writeln(sh);        // upto this, works, the same pointer 
 as set in
                          // c.dothings(), checked with write 
 instructions

      string[] si = *sh;
      writeln(si);
 }



 and then casted back, then i notice that it does not work. 
 Wondering why.
Sep 18 2014
parent "bearophile" <bearophileHUGS lycos.com> writes:
anonymous:


 Here, the pointer to the stack escapes the function. Don't do 
 that!


Hopefully the D type system will be improved with scoping 
tracking & management, to turn similar operations into 
compilation errors (as in Rust, but perhaps in a less refined 
way).

Bye,
bearophile
Sep 18 2014