www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - runtime evaluation

reply "hoya" <sighoya gmail.com> writes:
If I read it correctly, there is no possibility to create 
classes, structs, templates, functions or to import libs at 
runtime.
Is there already an approach to add some functionality?
I know, that D is a compiled language, but Java is too and 
supports class loading at runtime or more in general: runtime 
evaluation.
Dec 14 2013
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
hoya:


 If I read it correctly, there is no possibility to create 
 classes, structs, templates, functions or to import libs at 
 runtime.


Right.



 Is there already an approach to add some functionality?
 I know, that D is a compiled language, but Java is too and 
 supports class loading at runtime or more in general: runtime 
 evaluation.


Java often runs on a JVM, so it keeps its compiler around at 
run-time, unlike most cases in D.

With enough work you can do the same in D with the LLVM, and keep 
the compiler as a service at run-time, if you want even as a JIT. 
But currently ldc2 doesn't have this feature.

Currently probably you can synthesize D source code, save the 
source, call the D compiler from D, and use it as library.

Bye,
bearophile
Dec 14 2013
parent "hoya" <sighoya gmail.com> writes:
On Saturday, 14 December 2013 at 13:04:47 UTC, bearophile wrote:

 Currently probably you can synthesize D source code, save the 
 source, call the D compiler from D, and use it as library.

 Bye,
 bearophile


Thanks, bearophile. But how to do this, if it is not possible to 
import a library at runtime?
Dec 14 2013
prev sibling next sibling parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Sat, 14 Dec 2013 13:15:13 +0100
schrieb "hoya" <sighoya gmail.com>:


 If I read it correctly, there is no possibility to create 
 classes, structs, templates, functions or to import libs at 
 runtime.


You can load libraries at runtime like OpenGL or a database
driver. This applies to all C and D libraries as well as C++
libraries to some extent.


 Is there already an approach to add some functionality?
 I know, that D is a compiled language, but Java is too and 
 supports class loading at runtime or more in general: runtime 
 evaluation.


Did something change in recent Java versions? E.g. to get a
create a new class I needed the JDK around (not just the JRE)
for the compiler, which had to be added to the classpath first.
I wouldn't have considered that a 1st class language feature.
After all you compile your source code first with the optional
development tools and then load the resulting class file which
is not much different from calling the D compiler on some D
source to create a library and loading that.
Or does Java have an eval() method of some sort now?

-- 
Marco
Dec 14 2013
parent reply "hoya" <sighoya gmail.com> writes:
On Saturday, 14 December 2013 at 13:44:01 UTC, Marco Leise wrote:


 You can load libraries at runtime like OpenGL or a database
 driver. This applies to all C and D libraries as well as C++
 libraries to some extent.


Can you give me an example how to do this?

 Or does Java have an eval() method of some sort now?


Not directly, you can read this: 
http://rosettacode.org/wiki/Runtime_evaluation/In_an_environment#Java
Dec 14 2013
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Sat, 14 Dec 2013 14:59:42 +0100
schrieb "hoya" <sighoya gmail.com>:


 On Saturday, 14 December 2013 at 13:44:01 UTC, Marco Leise wrote:
 

 You can load libraries at runtime like OpenGL or a database
 driver. This applies to all C and D libraries as well as C++
 libraries to some extent.


 Can you give me an example how to do this?


Hmm, there are two options when it comes to dynamic linking.

You can directly refer to the functions you use from some
library. An example is std.net.curl in Phobos. You compile
your program with "dmd -L-lcurl <source>" to link to the curl
library which is written in C and common on Linux systems.
This is the simple case where you don't need to write any
extra code.

If you want to load a plugin at runtime or OpenGL (which is a
different library for each graphics card vendor) you will need
to use the dynamic linker (libdl.so) which handles this case:
"dmd -L-ldl <source>" and then use "dlopen" and "dlsym" in
your program to find the addresses of functions. Derelict 3 is
a good example of loading libraries at runtime this way:
https://github.com/aldacron/Derelict3
It is a wrapper/binding collection for many multimedia/game
related libraries and you can reuse its derelict.util module
to load your own libraries at runtime.

Which case are you interested in?


 Or does Java have an eval() method of some sort now?


 Not directly, you can read this: 
 http://rosettacode.org/wiki/Runtime_evaluation/In_an_environment#Java


But that is what I described above. A Java compiler is not a
requirement for a proper Java installation. It is an external
tool that they load and the expression evaluation is done
by writing the code to a Java source file on the file system,
compiling that and loading it. This can be done with dmd as
well, but in both cases it is crutch to bring eval()-like
functionality to a statically compiled language. :)

-- 
Marco
Dec 14 2013
parent reply "hoya" <sighoya gmail.com> writes:
On Saturday, 14 December 2013 at 17:47:34 UTC, Marco Leise wrote:

 Am Sat, 14 Dec 2013 14:59:42 +0100
 schrieb "hoya" <sighoya gmail.com>:


 On Saturday, 14 December 2013 at 13:44:01 UTC, Marco Leise 
 wrote:
 

 You can load libraries at runtime like OpenGL or a database
 driver. This applies to all C and D libraries as well as C++
 libraries to some extent.


 Can you give me an example how to do this?


 Hmm, there are two options when it comes to dynamic linking.

 You can directly refer to the functions you use from some
 library. An example is std.net.curl in Phobos. You compile
 your program with "dmd -L-lcurl <source>" to link to the curl
 library which is written in C and common on Linux systems.
 This is the simple case where you don't need to write any
 extra code.

 If you want to load a plugin at runtime or OpenGL (which is a
 different library for each graphics card vendor) you will need
 to use the dynamic linker (libdl.so) which handles this case:
 "dmd -L-ldl <source>" and then use "dlopen" and "dlsym" in
 your program to find the addresses of functions. Derelict 3 is
 a good example of loading libraries at runtime this way:
 https://github.com/aldacron/Derelict3
 It is a wrapper/binding collection for many multimedia/game
 related libraries and you can reuse its derelict.util module
 to load your own libraries at runtime.

 Which case are you interested in?


The Latter

 This can be done with dmd as
 well, but in both cases it is crutch to bring eval()-like
 functionality to a statically compiled language. :)


Hmm. An eval() method is good, if writing something like a Drepl. 
I've heard from someone that other people have tried some work on 
it.
Or good case, if you can use a dynamic mixin(). When read a 
string from input, then it is possible to create a structure like 
struct, class and so on. With dlopen() you need at least a 
library file, where all your code is written in, as I thin 
correctly. But with a dynamic mixin() you have the ability, to 
add structures in-memory. So this is more efficient.
Dec 14 2013
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Sat, 14 Dec 2013 21:11:46 +0100
schrieb "hoya" <sighoya gmail.com>:


 On Saturday, 14 December 2013 at 17:47:34 UTC, Marco Leise wrote:

 Am Sat, 14 Dec 2013 14:59:42 +0100
 schrieb "hoya" <sighoya gmail.com>:


 On Saturday, 14 December 2013 at 13:44:01 UTC, Marco Leise 
 wrote:
 

 You can load libraries at runtime like OpenGL or a database
 driver. This applies to all C and D libraries as well as C++
 libraries to some extent.


 Can you give me an example how to do this?


 Hmm, there are two options when it comes to dynamic linking.

 You can directly refer to the functions you use from some
 library. An example is std.net.curl in Phobos. You compile
 your program with "dmd -L-lcurl <source>" to link to the curl
 library which is written in C and common on Linux systems.
 This is the simple case where you don't need to write any
 extra code.

 If you want to load a plugin at runtime or OpenGL (which is a
 different library for each graphics card vendor) you will need
 to use the dynamic linker (libdl.so) which handles this case:
 "dmd -L-ldl <source>" and then use "dlopen" and "dlsym" in
 your program to find the addresses of functions. Derelict 3 is
 a good example of loading libraries at runtime this way:
 https://github.com/aldacron/Derelict3
 It is a wrapper/binding collection for many multimedia/game
 related libraries and you can reuse its derelict.util module
 to load your own libraries at runtime.

 Which case are you interested in?


 The Latter

 This can be done with dmd as
 well, but in both cases it is crutch to bring eval()-like
 functionality to a statically compiled language. :)


 Hmm. An eval() method is good, if writing something like a Drepl. 
 I've heard from someone that other people have tried some work on 
 it.
 Or good case, if you can use a dynamic mixin(). When read a 
 string from input, then it is possible to create a structure like 
 struct, class and so on. With dlopen() you need at least a 
 library file, where all your code is written in, as I thin 
 correctly. But with a dynamic mixin() you have the ability, to 
 add structures in-memory. So this is more efficient.


A simple REPL that doesn't remember any previous state is
easily written today. It becomes difficult when you want to
keep your state from command to command or try to delete
previous definitions.
For what you want there would have to be a JIT compiler in
every D program in the first place. Interpreted languages and
JIT compiled languages have an advantage here.
eval() is more efficient in the way that you don't need to go
though an external file and have the dynamic linker load it,
yes. But even if a real eval() existed, there is no point in
adding structs to the program that way, since the rest of the
code isn't prepared to accept them. Sub-classes would work.
I would look into using an interpreted or JIT compiled
language from within D for the parts of your code that need
to be dynamic. Take like LuaD for example:
https://github.com/JakobOvrum/LuaD
Python or JavaScript bindings would do as well.

-- 
Marco
Dec 14 2013
parent "hoya" <sighoya gmail.com> writes:
On Saturday, 14 December 2013 at 23:36:12 UTC, Marco Leise wrote:

 Am Sat, 14 Dec 2013 21:11:46 +0100
 schrieb "hoya" <sighoya gmail.com>:


 On Saturday, 14 December 2013 at 17:47:34 UTC, Marco Leise 
 wrote:

 Am Sat, 14 Dec 2013 14:59:42 +0100
 schrieb "hoya" <sighoya gmail.com>:


 On Saturday, 14 December 2013 at 13:44:01 UTC, Marco Leise 
 wrote:
 

 You can load libraries at runtime like OpenGL or a 
 database
 driver. This applies to all C and D libraries as well as 
 C++
 libraries to some extent.


 Can you give me an example how to do this?


 Hmm, there are two options when it comes to dynamic linking.

 You can directly refer to the functions you use from some
 library. An example is std.net.curl in Phobos. You compile
 your program with "dmd -L-lcurl <source>" to link to the curl
 library which is written in C and common on Linux systems.
 This is the simple case where you don't need to write any
 extra code.

 If you want to load a plugin at runtime or OpenGL (which is a
 different library for each graphics card vendor) you will 
 need
 to use the dynamic linker (libdl.so) which handles this case:
 "dmd -L-ldl <source>" and then use "dlopen" and "dlsym" in
 your program to find the addresses of functions. Derelict 3 
 is
 a good example of loading libraries at runtime this way:
 https://github.com/aldacron/Derelict3
 It is a wrapper/binding collection for many multimedia/game
 related libraries and you can reuse its derelict.util module
 to load your own libraries at runtime.

 Which case are you interested in?


 The Latter

 This can be done with dmd as
 well, but in both cases it is crutch to bring eval()-like
 functionality to a statically compiled language. :)


 Hmm. An eval() method is good, if writing something like a 
 Drepl. I've heard from someone that other people have tried 
 some work on it.
 Or good case, if you can use a dynamic mixin(). When read a 
 string from input, then it is possible to create a structure 
 like struct, class and so on. With dlopen() you need at least 
 a library file, where all your code is written in, as I thin 
 correctly. But with a dynamic mixin() you have the ability, to 
 add structures in-memory. So this is more efficient.


 A simple REPL that doesn't remember any previous state is
 easily written today. It becomes difficult when you want to
 keep your state from command to command or try to delete
 previous definitions.
 For what you want there would have to be a JIT compiler in
 every D program in the first place. Interpreted languages and
 JIT compiled languages have an advantage here.
 eval() is more efficient in the way that you don't need to go
 though an external file and have the dynamic linker load it,
 yes. But even if a real eval() existed, there is no point in
 adding structs to the program that way, since the rest of the
 code isn't prepared to accept them. Sub-classes would work.
 I would look into using an interpreted or JIT compiled
 language from within D for the parts of your code that need
 to be dynamic. Take like LuaD for example:
 https://github.com/JakobOvrum/LuaD
 Python or JavaScript bindings would do as well.


I will keep this in mind, thanks.
Thanks all for helping!
Dec 15 2013
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-12-14 13:15, hoya wrote:

 If I read it correctly, there is no possibility to create classes,
 structs, templates, functions or to import libs at runtime.
 Is there already an approach to add some functionality?
 I know, that D is a compiled language, but Java is too and supports
 class loading at runtime or more in general: runtime evaluation.


You can instantiate classes at runtime using Object.factory. You can 
also loaded classes using dynamic libraries, just as in C or C++.

-- 
/Jacob Carlborg
Dec 14 2013
parent reply "hoya" <sighoya gmail.com> writes:
On Saturday, 14 December 2013 at 13:45:00 UTC, Jacob Carlborg 
wrote:

 You can instantiate classes at runtime using Object.factory. 
 You can also loaded classes using dynamic libraries, just as in 
 C or C++.


Yes, but loading a dynamic lib (shared) in C/C++ is at compile 
type, if I remembering right. But is there a way in D like 
dlopen() in C, which can be used at runtime?
Dec 14 2013
parent Mike Parker <aldacron gmail.com> writes:
On 12/14/2013 11:07 PM, hoya wrote:

 On Saturday, 14 December 2013 at 13:45:00 UTC, Jacob Carlborg wrote:

 You can instantiate classes at runtime using Object.factory. You can
 also loaded classes using dynamic libraries, just as in C or C++.


 Yes, but loading a dynamic lib (shared) in C/C++ is at compile type, if
 I remembering right. But is there a way in D like dlopen() in C, which
 can be used at runtime?


That's what Jacob is talking about. D can interface with C libraries 
just fine, so dlopen() is available on Posix systems and LoadLibrary on 
Windows. That's the approach I use with the C library bindings in 
Derelict[1].

Shared libraries are never loaded at compile time. You can link with an 
import library (on Windows) or directly with the shared object, but that 
just sets the executable up so that the library is loaded by the OS when 
the app starts up. dlopen and friends eliminate the need for a link 
step, but require more effort to load manually.

[1] https://github.com/DerelictOrg
Dec 14 2013