www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Calling function within class.

reply Vino <akashvino79 gmail.com> writes:
Hi All,

   Request your help on how to call a function(listFile) from 
another function(getFilelist) within the same class(GetDirlist), 
below is an example code.

Code:

class GetDirlist {

 system private auto listFile(immutable string st)
{
  auto fl = execute(["ls","-l"]);
  enforce(fl.status == 0, format("File not Found: %s", fl.status));
  return fl.output;
}

 system public auto getFilelist()
{
   Array!string amidata;
   auto flstore = Array!string("/path1/Dir1", "path2/Dir2");
   foreach(st; parallel(flstore,1)) {
      auto fltask = task!listFile(st);
      fltask.executeInNewThread();
      amidata.insert(fltask.yieldForce);
   }
   return amidata[];
}
}

void main() {
writeln(getFilelist());
}

From,
Vino.B
Nov 18 2020
next sibling parent reply frame <frame86 live.com> writes:
On Wednesday, 18 November 2020 at 15:01:53 UTC, Vino wrote:

 Hi All,

   Request your help on how to call a function(listFile) from 
 another function(getFilelist) within the same 
 class(GetDirlist), below is an example code.


I think it's basically the same issue like that recently opened 
topic:

https://forum.dlang.org/thread/ddxasuvusgibppcclsop forum.dlang.org

You need to pass your class object as argument to a static method 
or anonymous function.
Nov 18 2020
parent reply Vino <akashvino79 gmail.com> writes:
On Wednesday, 18 November 2020 at 16:53:44 UTC, frame wrote:

 On Wednesday, 18 November 2020 at 15:01:53 UTC, Vino wrote:

 Hi All,

   Request your help on how to call a function(listFile) from 
 another function(getFilelist) within the same 
 class(GetDirlist), below is an example code.


 I think it's basically the same issue like that recently opened 
 topic:

 https://forum.dlang.org/thread/ddxasuvusgibppcclsop forum.dlang.org

 You need to pass your class object as argument to a static 
 method or anonymous function.


Hi,

   I made the changes as below , still not working

auto fltask = task!listFile(st);
to
auto fltask = task!({listFile(st);})(this).executeInNewThread();

Error:
/DTECH/LDC/dlang/ldc-1.24.0/bin/../import/std/parallelism.d(516,34): Error:
function literal __lambda2() is not callable using argument types (GetDirlist)
/DTECH/LDC/dlang/ldc-1.24.0/bin/../import/std/parallelism.d(516,34):       
expected 0 argument(s), not 1
/DTECH/LDC/dlang/ldc-1.24.0/bin/../import/std/parallelism.d(842,16): Error:
template instance GetDirlist.GetDirlist.getFilelist.__foreachbody2.
Task!(delegate ()  system { this.listFile(st); } , GetDirlist) 
error instantiating instantiated from here: task!(delegate () 
 system { this.listFile(st); } , GetDirlist)

From,
Vino.B
Nov 18 2020
parent reply frame <frame86 live.com> writes:
On Wednesday, 18 November 2020 at 17:55:36 UTC, Vino wrote:


   I made the changes as below , still not working

 auto fltask = task!listFile(st);
 to
 auto fltask = task!({listFile(st);})(this).executeInNewThread();


The syntax is just wrong:

auto fltask = task!({listFile(st);})(this).executeInNewThread();

Should be:

auto fltask = task!((GetDirlist 
obj){obj.listFile(st);})(this).executeInNewThread();

It's the anonymous function syntax in one line compressed - it's 
equal to:

auto fltask = task!(
   function(GetDirlist obj) {
      obj.listFile(st);
   }
)(this).executeInNewThread();


But with your context it must look like this:

auto fltask = task!(
   function(GetDirlist obj, string foo) {
      obj.listFile(foo);
   }
)(this, st);

Still your code need to be fixed at lot to get working.
Nov 18 2020
parent reply Vino <akashvino79 gmail.com> writes:
On Wednesday, 18 November 2020 at 18:24:59 UTC, frame wrote:

 On Wednesday, 18 November 2020 at 17:55:36 UTC, Vino wrote:


   I made the changes as below , still not working

 auto fltask = task!listFile(st);
 to
 auto fltask = 
 task!({listFile(st);})(this).executeInNewThread();


 The syntax is just wrong:

 auto fltask = task!({listFile(st);})(this).executeInNewThread();

 Should be:

 auto fltask = task!((GetDirlist 
 obj){obj.listFile(st);})(this).executeInNewThread();

 It's the anonymous function syntax in one line compressed - 
 it's equal to:

 auto fltask = task!(
   function(GetDirlist obj) {
      obj.listFile(st);
   }
 )(this).executeInNewThread();


 But with your context it must look like this:

 auto fltask = task!(
   function(GetDirlist obj, string foo) {
      obj.listFile(foo);
   }
 )(this, st);

 Still your code need to be fixed at lot to get working.


Hi,

   The above code is a sample code, but the logic is same, correct 
me if my understanding is wrong, in the above code "obj" is a an 
object for the class GetDirlist, so we are accessing the class 
member using "obj.listFile(st)" , so why do we need the "(this)" 
and also why is this so complicated in D where as in PHP it is 
simple like below

PHP Code:
class PHPclass {
       function test1 ($st) { return $st; }
       function test2 ()  { return $this->test1("Test"); }
}

$obj = new PHPclass();
print_r($obj->test2());
Nov 18 2020
next sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 11/18/20 11:25 AM, Vino wrote:


 why is this
 so complicated in D where as in PHP it is simple like below

 PHP Code:
 class PHPclass {
        function test1 ($st) { return $st; }
        function test2 ()  { return $this->test1("Test"); }
 }

 $obj = new PHPclass();
 print_r($obj->test2());


It is so similar in D that one can say they are the same:

class PHPclass {
   auto test1(T)(T st) { return st; }
   auto test2()  { return this.test1("Test"); }
}

import std.stdio;

void main() {
   auto obj = new PHPclass();
   writeln(obj.test2());
}

Ali
Nov 18 2020
prev sibling parent frame <frame86 live.com> writes:
On Wednesday, 18 November 2020 at 19:25:06 UTC, Vino wrote:


   The above code is a sample code, but the logic is same, 
 correct me if my understanding is wrong, in the above code 
 "obj" is a an object for the class GetDirlist, so we are 
 accessing the class member using "obj.listFile(st)" , so why do 
 we need the "(this)" and also why is this so complicated in D 
 where as in PHP it is simple like below


You need to understand that you cannot use "this" in a function 
that you call in a new thread but task! starts a kind of new 
thread. That's not allowed by the runtime.

Your PHP code does not show any threading and PHP does not 
support threading without extensions. Also PHP does not allow 
$this for real threading extensions like parallel.

For D that means...

// before that statement you can call "this" - it points to your 
GetDirlist class.

auto fltask = task!(

    // this function is called in the new thread
    // and it's enclosed in the body of task! but could be also 
just a pointer to another
    // function. This is valid as long as the compiler do not need 
to access outside
    // variables/scope. "this" would also access the outside scope.

    function(GetDirlist obj, string st) {
             obj.listFile(st);
    }

    // end of function and thread
)

// after that statement you can call "this" again
// so we pass "this" (your object) as argument to the function 
that cannot access "this"
// and the string that the function is expecting

(this, "foo");
Nov 19 2020
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 11/18/20 7:01 AM, Vino wrote:


    Request your help on how to call a function(listFile) from another
 function(getFilelist) within the same class(GetDirlist), below is an
 example code.


That code looks unnecessarily complex to me. First of all, parallel() 
already executes the loop body in separate threads, so I don't see any 
reason for Task in that code.

std.parallel will appear during my DConf presentation on Saturday. The 
following program applies ideas from some of my slides and just works in 
parallel:

import std.process;
import std.exception;
import std.format;
import std.parallelism;
import std.stdio;

class GetDirlist {

    system private auto listFile(immutable string st)
   {
     auto fl = execute(["ls","-l"]);
     enforce(fl.status == 0, format("File not Found: %s", fl.status));
     return fl.output;
   }

    system public auto getFilelist()
   {
     // I am using D dynamic arrays for simpliticy
     auto flstore = [ "/path1/Dir1", "path2/Dir2" ];

     // Note preallocated slot for each result:
     auto amidata = new string[flstore.length];

     // Taking advantage of automatic loop counter
     foreach(i, st; parallel(flstore,1)) {
       // Each execution assigns to its own slot
       amidata[i] = listFile(st);
     }
     return amidata[];
   }
}

void main() {
   // Need an object to call a non-static member function:
   auto g = new GetDirlist();
   writeln(g.getFilelist());
}

Ali
Nov 18 2020
parent reply Vino <akashvino79 gmail.com> writes:
On Wednesday, 18 November 2020 at 21:33:58 UTC, Ali Çehreli wrote:

 On 11/18/20 7:01 AM, Vino wrote:


    Request your help on how to call a function(listFile) from


 another

 function(getFilelist) within the same class(GetDirlist),


 below is an

 example code.


 That code looks unnecessarily complex to me. First of all, 
 parallel() already executes the loop body in separate threads, 
 so I don't see any reason for Task in that code.

 std.parallel will appear during my DConf presentation on 
 Saturday. The following program applies ideas from some of my 
 slides and just works in parallel:

 import std.process;
 import std.exception;
 import std.format;
 import std.parallelism;
 import std.stdio;

 class GetDirlist {

    system private auto listFile(immutable string st)
   {
     auto fl = execute(["ls","-l"]);
     enforce(fl.status == 0, format("File not Found: %s", 
 fl.status));
     return fl.output;
   }

    system public auto getFilelist()
   {
     // I am using D dynamic arrays for simpliticy
     auto flstore = [ "/path1/Dir1", "path2/Dir2" ];

     // Note preallocated slot for each result:
     auto amidata = new string[flstore.length];

     // Taking advantage of automatic loop counter
     foreach(i, st; parallel(flstore,1)) {
       // Each execution assigns to its own slot
       amidata[i] = listFile(st);
     }
     return amidata[];
   }
 }

 void main() {
   // Need an object to call a non-static member function:
   auto g = new GetDirlist();
   writeln(g.getFilelist());
 }

 Ali


Hi Ali,

Thank you very much, your solution works for my example, but it 
does not work for the main goal, let me explain what we are 
trying to perform.

Nut shell: Try to execute an aws command on sever accounts in 
parallel to get some data.

Noe: each account has as separate username and password store in 
a database table(encrypted).

Cod Logic
Fetch the username/ password from the table for each account.

Get the “awssecrete” key and “accesskey” for each account by 
calling an aws api using the above username/password.

Set the fetched key’s as an environment variable.

Execute the aws command and get the data for each of the account

   As we have many accounts what we are trying is to get the data 
in parallel(execute the aws command in parallel for each account 
and store the result in a array). At present our code is working 
fine(without parallel), the moment we enable parallelism, it is 
throwing an error on the SQL part (Fetch the username/ password 
from the table for each account), as it could not execute the SQL 
query in parallel for different account. If there is any other 
logic please do let me know will give it a try. Below is the SQL 
code.

 trusted public auto getAwsconf(immutable string account)
{
  auto con = new GetConnections();
  Statement stmt = con.db.prepare("SELECT 
username,AES_DECRYPT(b.userpass,b.key,b.vector) AS passwd FROM 
config WHERE account = :account");
  stmt.setParameter("account", account);
  RowSet awsaccount = stmt.query();
  scope(exit) con.db.close();
  return awsaccount;
}

From,
Vino.B
Nov 19 2020
next sibling parent frame <frame86 live.com> writes:
On Thursday, 19 November 2020 at 19:51:24 UTC, Vino wrote:
the moment we enable

 parallelism, it is throwing an error on the SQL part (Fetch the 
 username/ password from the table for each account), as it 
 could not execute the SQL query in parallel for different 
 account.


I see no reason for a SELECT call (UPDATE/INSERT could cause 
locks...). Which error do you get?
Nov 19 2020
prev sibling parent Arafel <er.krali gmail.com> writes:
On 19/11/20 20:51, Vino wrote:

 Hi Ali,
 
 Thank you very much, your solution works for my example, but it does not 
 work for the main goal, let me explain what we are trying to perform.
 
 Nut shell: Try to execute an aws command on sever accounts in parallel 
 to get some data.
 
 Noe: each account has as separate username and password store in a 
 database table(encrypted).
 
 Cod Logic
 Fetch the username/ password from the table for each account.
 
 Get the “awssecrete” key and “accesskey” for each account by calling
an 
 aws api using the above username/password.
 
 Set the fetched key’s as an environment variable.
 
 Execute the aws command and get the data for each of the account
 
    As we have many accounts what we are trying is to get the data in 
 parallel(execute the aws command in parallel for each account and store 
 the result in a array). At present our code is working fine(without 
 parallel), the moment we enable parallelism, it is throwing an error on 
 the SQL part (Fetch the username/ password from the table for each 
 account), as it could not execute the SQL query in parallel for 
 different account. If there is any other logic please do let me know 
 will give it a try. Below is the SQL code.
 
  trusted public auto getAwsconf(immutable string account)
 {
   auto con = new GetConnections();
   Statement stmt = con.db.prepare("SELECT 
 username,AES_DECRYPT(b.userpass,b.key,b.vector) AS passwd FROM config 
 WHERE account = :account");
   stmt.setParameter("account", account);
   RowSet awsaccount = stmt.query();
   scope(exit) con.db.close();
   return awsaccount;
 }
 
 From,
 Vino.B


Hi,

How does the `GetConnections()` constructor work? Is it creating a new 
connection to the DB each time it's called, or is it getting them from a 
pool? In any case it should probably return a different connection for 
each thread (so the pool size should be at least the same as the number 
of workers).

Otherwise one thread will try to start a new SQL operation while there 
is another already running. In the best case, if the library is 
thread-safe, it could enqueue them thus negating the benefit of 
parallelism... if not, then you'll likely get errors like the ones 
you're seeing.

Best,

A.
Nov 20 2020