www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - MiniD 2 - Tentative Release Candidate 1

reply "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
ffs, if I don't release it now I'll _never_ release it.

After over a year of work, a few drastic changes, and a completely new
implementation, I give you: MiniD 2.

http://dsource.org/projects/minid

---------------------------------
What's new in the language?

- Lots.

- Several syntactic changes and additions leading to much
cleaner-looking, terser code.

- Classes and instances have been greatly improved, thanks to a
one-year trip into the land of prototype-based objects.

- More metamethods for more object customizability.  In particular,
field access and method calls can be intercepted, and there is now a
full complement of arithmetic operator overloads (including reverse
versions).

- Weak references.

- A new type which can hold instances of D classes, for use by native code.

- Sixty-four (64) bit integers.  What will *you* do with all those bits?

- A much-expanded and improved standard library.

- Much more well-defined semantics for most language constructs and operations.

- A more flexible module loading system.

---------------------------------
What's this new implementation?

- It's waaaaaaaaaAAAy better than the old one.  This is really the
killer feature of MiniD 2.

- It has its own garbage collector, meaning that your scripts won't
trigger native app GC cycles (well.. much, much less often), and MiniD
code also runs much faster.

- It has a completely new API that is similar to the Lua or Squirrel APIs.

- There Is One Way To Do It.  The old implementation and API were
hole-y and inconsistent.  Now, you're pretty much locked into doing
things one way, meaning much less chance for error and much less to
learn.

- Virtually anything you can do in the language, you can do in the API
without much difficulty.  The new API is much more well-rounded than
the old one.

- Performancewise, it beats the pants off of Python, though that's not
_necessarily_ saying much.  I don't know how it measures up to Lua as
I don't have a way of measuring Lua's performance from within the
language (so as to exclude startup and compilation times), but I get
the impression that it's pretty close in some cases.

---------------------------------
What do I need to compile it?

- Tango, SVN revision 4048.  I *told* you this was a tentative release
candidate.  I don't know if it'll work with any older/newer ones.

- DMD 1.034 or newer, or equivalent.  LDC seems to work as well.  GDC
does not.  Sorry Mac users, go get David to work on GDC again ;)

- Some kind of build tool, DSSS/rebuild or bu[il]d.

- MiniD itself.  Just check out the head revision of the MiniD trunk
at http://svn.dsource.org/projects/minid/trunk.

- The understanding that if you are on Linux and try to do all sorts
of stupid fancy stuff with symbolic links and putting things in
nonstandard directories and making directories read-only, I will not
have any sympathy for you when you can't get D, Tango, DSSS, or MiniD
to work correctly.  The D toolchain is fragile enough as it is, please
don't bend it.

---------------------------------
Why am I calling this a tentative release candidate?

- Some aspects of the library (THE BINDING LIBRARY, which is much
better than MiniD 1's, and the compiler to an extent) have not been
very heavily tested and are likely to be somewhat buggy.  But I'm one
guy with limited time and so if you could start using it and making it
fail that'd be just great.

- Some of the docs are not yet complete.  The section on modules is
not yet complete, but not a whole lot has changed there anyway.  The
DDocs for the individual API modules are basically complete but are
somewhat lacking in more complex examples, which is what the tutorials
are supposed to be for.

- I haven't compiled it on Linux recently.  Yeah, I should do that
more often.  But it worked last month or so.

---------------------------------
What is needed before an "official" release is made?

- Time, something I don't have a lot of this week.  After that, I have
three weeks of bliss.

- Testing.

- Finished docs.

- Maybe a couple more additions/changes.

---------------------------------
Anything else?

- Mudkip.
Dec 07 2008
next sibling parent reply mpt <foo invalid.x> writes:
Great stuff, but I have some trouble with the binding lib.

1. If I don't wrap a constructor, I get an error:
minid/bind.d(1101): static assert  "Cannot call default constructor for class
Foo; please wrap a constructor explicitly"

2. If I wrap a constructor with no parameters (i.e. WrapCtors!(void
function())), I get an error:
minid/bind.d(1066): constructor
minid.bind.WrappedClass!(Foo,"Foo","",WrapCtors!(void
function()),WrapMethod!(Call)).WrappedClass.ClassCtorShims!(void
function()).this (MDVM*,()) does not match parameter types ()
minid/bind.d(1066): Error: expected 1 arguments, not 0

3. I can't find an obvious way to initialize global variables, so I try
superPush, which gives a runtime error:
minid.types.MDException: <no location available>: newInstance - expected
'class' for base, not 'null'

Test code below:

import minid.bind;
import minid.api;

class Foo {
    this() {}
    this(int i) {}
    void Call() {}
}

void main() {
    MDVM vm;
    auto t = openVM(&vm);
    loadStdlibs(t);
    WrapGlobals!(
        WrapType!(
            Foo, "Foo",
            WrapCtors!(void function(int)),
            WrapMethod!(Foo.Call)
        )
    )(t);
    superPush(t, new Foo());
    newGlobal(t, "myfoo");
}
Dec 08 2008
parent reply "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Mon, Dec 8, 2008 at 3:44 AM, mpt <foo invalid.x> wrote:

 Great stuff, but I have some trouble with the binding lib.


Good!  Because I wasn't ;)


 1. If I don't wrap a constructor, I get an error:
 minid/bind.d(1101): static assert  "Cannot call default constructor for class
Foo; please wrap a constructor explicitly"


Fixed.


 2. If I wrap a constructor with no parameters (i.e. WrapCtors!(void
function())), I get an error:
 minid/bind.d(1066): constructor
minid.bind.WrappedClass!(Foo,"Foo","",WrapCtors!(void
function()),WrapMethod!(Call)).WrappedClass.ClassCtorShims!(void
function()).this (MDVM*,()) does not match parameter types ()
 minid/bind.d(1066): Error: expected 1 arguments, not 0


Fixed.


 3. I can't find an obvious way to initialize global variables, so I try
superPush, which gives a runtime error:


You're actually doing it right, unless you wanted to instantiate the
wrapped MiniD class, in which case it'd be more like:

   pushGlobal(t, "Foo"); // get the global class Foo
   pushNull(t); // make room for 'this'
   rawCall(t, -2, 1); // call the class to instantiate it (1 return
for the instance)
   newGlobal(t, "myfoo"); // save it in the global myfoo



 minid.types.MDException: <no location available>: newInstance - expected
'class' for base, not 'null'


Fixed.


 Test code below:

 import minid.bind;
 import minid.api;

 class Foo {
    this() {}
    this(int i) {}
    void Call() {}
 }

 void main() {
    MDVM vm;
    auto t = openVM(&vm);
    loadStdlibs(t);
    WrapGlobals!(
        WrapType!(
            Foo, "Foo",
            WrapCtors!(void function(int)),
            WrapMethod!(Foo.Call)
        )
    )(t);
    superPush(t, new Foo());
    newGlobal(t, "myfoo");
 }


This code now works, just svn up your MiniD installation :)
Dec 08 2008
parent reply mpt <mpt iki.fi> writes:
Jarrett Billingsley wrote:

 On Mon, Dec 8, 2008 at 3:44 AM, mpt <foo invalid.x> wrote:

 Great stuff, but I have some trouble with the binding lib.


 
 Good!  Because I wasn't ;)
 

 1. If I don't wrap a constructor, I get an error:
 minid/bind.d(1101): static assert  "Cannot call default constructor for class
Foo; please wrap a constructor explicitly"


 
 Fixed.


Fix compiles for the explicitly defined ctor, but not for the default. 
This code gives:
minid/bind.d(1110): Error: no super class constructor for Foo

import minid.bind;
import minid.api;

class Foo {}

void main() {
     MDVM vm;
     auto t = openVM(&vm);
     loadStdlibs(t);
     WrapGlobals!(
         WrapType!(
             Foo, "Foo"
         )
     )(t);
}


 3. I can't find an obvious way to initialize global variables, so I try
superPush, which gives a runtime error:


 
 You're actually doing it right, unless you wanted to instantiate the
 wrapped MiniD class, in which case it'd be more like:
 
    pushGlobal(t, "Foo"); // get the global class Foo
    pushNull(t); // make room for 'this'
    rawCall(t, -2, 1); // call the class to instantiate it (1 return
 for the instance)
    newGlobal(t, "myfoo"); // save it in the global myfoo
 


I can't see the difference between superPush-created myfoo and one 
created by your example. superPush doc says it creates MiniD-converted 
objects, but they're not usable.

superPush(t, new Foo());
newGlobal(t, "myfoo");
runString(t, "myfoo.Call();");

This gives: 
"tango.core.Exception.AssertException ../extern/minid/minid/bind.d(1397): 
Invalid 'this' parameter passed to method Foo.Call"

Your example works, but it's not what I need. I want to create the Foo 
object in D myself and pass it to MiniD and call its methods.

Ideally, I'd like to have/write a function like
void setGlobal(Type)(MDThread* t, char[] name, Type value) to create and 
modify global variables.
Dec 08 2008
parent "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Mon, Dec 8, 2008 at 3:22 PM, mpt <mpt iki.fi> wrote:

 Fix compiles for the explicitly defined ctor, but not for the default. This
 code gives:
 minid/bind.d(1110): Error: no super class constructor for Foo

 import minid.bind;
 import minid.api;

 class Foo {}

 void main() {
    MDVM vm;
    auto t = openVM(&vm);
    loadStdlibs(t);
    WrapGlobals!(
        WrapType!(
            Foo, "Foo"
        )
    )(t);
 }


Fixed.


 I can't see the difference between superPush-created myfoo and one created
 by your example. superPush doc says it creates MiniD-converted objects, but
 they're not usable.


There shouldn't be much different, at least from the point of view of
the MiniD code.  If you want to know, the difference is that instances
of the wrapped class created using MiniD are actually instances of a
class that is _derived_ from the class you wrapped, whereas instances
of the class you push are instances of the original class.  But the
binding library is *supposed* to abstract that away ;)


 superPush(t, new Foo());
 newGlobal(t, "myfoo");
 runString(t, "myfoo.Call();");

 This gives:
 "tango.core.Exception.AssertException ../extern/minid/minid/bind.d(1397):
 Invalid 'this' parameter passed to method Foo.Call"


Fixed.


 Your example works, but it's not what I need. I want to create the Foo
 object in D myself and pass it to MiniD and call its methods.

 Ideally, I'd like to have/write a function like
 void setGlobal(Type)(MDThread* t, char[] name, Type value) to create and
 modify global variables.


The implementation of that might be:

void SetGlobal(Type)(MDThread* t, char[] name, Type value)
{
    superPush(t, value);

    if(findGlobal(t, name))
    {
        swap(t);
        fielda(t, -2, name);
        pop(t);
    }
    else
        newGlobal(t, name);
}

This will create a global if it doesn't already exist, and will update
an existing global.  If you want it to always update an existing
global, it simply becomes "superPush(t, value); setGlobal(t, name);".
Dec 08 2008
prev sibling parent reply davidl <davidl 126.com> writes:
I've learned a bit minid today.

I'm curious why foreach opApply don't use the yield instead of the current
implementation?
and another curious thing is how can I serialize/deserialize minid object in
minid?
any .tupleof for class available?

在 Mon, 08 Dec 2008 13:50:47 +0800，Jarrett Billingsley
<jarrett.billingsley gmail.com> 写道:


 ffs, if I don't release it now I'll _never_ release it.

 After over a year of work, a few drastic changes, and a completely new
 implementation, I give you: MiniD 2.

 http://dsource.org/projects/minid

 ---------------------------------
 What's new in the language?

 - Lots.

 - Several syntactic changes and additions leading to much
 cleaner-looking, terser code.

 - Classes and instances have been greatly improved, thanks to a
 one-year trip into the land of prototype-based objects.

 - More metamethods for more object customizability.  In particular,
 field access and method calls can be intercepted, and there is now a
 full complement of arithmetic operator overloads (including reverse
 versions).

 - Weak references.

 - A new type which can hold instances of D classes, for use by native code.

 - Sixty-four (64) bit integers.  What will *you* do with all those bits?

 - A much-expanded and improved standard library.

 - Much more well-defined semantics for most language constructs and operations.

 - A more flexible module loading system.

 ---------------------------------
 What's this new implementation?

 - It's waaaaaaaaaAAAy better than the old one.  This is really the
 killer feature of MiniD 2.

 - It has its own garbage collector, meaning that your scripts won't
 trigger native app GC cycles (well.. much, much less often), and MiniD
 code also runs much faster.

 - It has a completely new API that is similar to the Lua or Squirrel APIs.

 - There Is One Way To Do It.  The old implementation and API were
 hole-y and inconsistent.  Now, you're pretty much locked into doing
 things one way, meaning much less chance for error and much less to
 learn.

 - Virtually anything you can do in the language, you can do in the API
 without much difficulty.  The new API is much more well-rounded than
 the old one.

 - Performancewise, it beats the pants off of Python, though that's not
 _necessarily_ saying much.  I don't know how it measures up to Lua as
 I don't have a way of measuring Lua's performance from within the
 language (so as to exclude startup and compilation times), but I get
 the impression that it's pretty close in some cases.

 ---------------------------------
 What do I need to compile it?

 - Tango, SVN revision 4048.  I *told* you this was a tentative release
 candidate.  I don't know if it'll work with any older/newer ones.

 - DMD 1.034 or newer, or equivalent.  LDC seems to work as well.  GDC
 does not.  Sorry Mac users, go get David to work on GDC again ;)

 - Some kind of build tool, DSSS/rebuild or bu[il]d.

 - MiniD itself.  Just check out the head revision of the MiniD trunk
 at http://svn.dsource.org/projects/minid/trunk.

 - The understanding that if you are on Linux and try to do all sorts
 of stupid fancy stuff with symbolic links and putting things in
 nonstandard directories and making directories read-only, I will not
 have any sympathy for you when you can't get D, Tango, DSSS, or MiniD
 to work correctly.  The D toolchain is fragile enough as it is, please
 don't bend it.

 ---------------------------------
 Why am I calling this a tentative release candidate?

 - Some aspects of the library (THE BINDING LIBRARY, which is much
 better than MiniD 1's, and the compiler to an extent) have not been
 very heavily tested and are likely to be somewhat buggy.  But I'm one
 guy with limited time and so if you could start using it and making it
 fail that'd be just great.

 - Some of the docs are not yet complete.  The section on modules is
 not yet complete, but not a whole lot has changed there anyway.  The
 DDocs for the individual API modules are basically complete but are
 somewhat lacking in more complex examples, which is what the tutorials
 are supposed to be for.

 - I haven't compiled it on Linux recently.  Yeah, I should do that
 more often.  But it worked last month or so.

 ---------------------------------
 What is needed before an "official" release is made?

 - Time, something I don't have a lot of this week.  After that, I have
 three weeks of bliss.

 - Testing.

 - Finished docs.

 - Maybe a couple more additions/changes.

 ---------------------------------
 Anything else?

 - Mudkip.
Dec 09 2008
parent reply "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Tue, Dec 9, 2008 at 3:58 AM, davidl <davidl 126.com> wrote:

 I've learned a bit minid today.

 I'm curious why foreach opApply don't use the yield instead of the current
implementation?


Ah, but you can.  Coroutines have an opApply, and are therefore
iterable using foreach.  See the section on coroutines
(http://www.dsource.org/projects/minid/wiki/LanguageSpec2/Functions#Coroutines)
for some info on that.  Using coroutines for iteration is more
powerful, but of course the price you pay is that they're a lot more
expensive than a function closure.


 and another curious thing is how can I serialize/deserialize minid object in
minid?
 any .tupleof for class available?


Sure, have a look of fieldsOf/allFieldsOf in the base library
(http://www.dsource.org/projects/minid/wiki/StdLib2/BaseLib).  They'll
give you functions as well, but it's easy to skip over them.
Dec 09 2008
parent reply davidl <davidl 126.com> writes:
在 Tue, 09 Dec 2008 21:07:46 +0800，Jarrett Billingsley
<jarrett.billingsley gmail.com> 写道:


 On Tue, Dec 9, 2008 at 3:58 AM, davidl <davidl 126.com> wrote:

 I've learned a bit minid today.

 I'm curious why foreach opApply don't use the yield instead of the current
implementation?


 Ah, but you can.  Coroutines have an opApply, and are therefore
 iterable using foreach.  See the section on coroutines
 (http://www.dsource.org/projects/minid/wiki/LanguageSpec2/Functions#Coroutines)
 for some info on that.  Using coroutines for iteration is more
 powerful, but of course the price you pay is that they're a lot more
 expensive than a function closure.


I got an impression that a scripting lang coroutine is less expensive than a
function call? I guess coroutine in a scripting interpreted lang won't require
any thread involved, and the suspension of the execution maybe even more
costless compared to a return statement and reexecution of that function in the
case of opApply.
And I'm still not able to get opApply to work with coroutines, any example
available?
I always get the error message of "Iterated coroutine must be in the initial
state", I've no clue about what it's going on.


 and another curious thing is how can I serialize/deserialize minid object in
minid?
 any .tupleof for class available?


 Sure, have a look of fieldsOf/allFieldsOf in the base library
 (http://www.dsource.org/projects/minid/wiki/StdLib2/BaseLib).  They'll
 give you functions as well, but it's easy to skip over them.
 


thanks, that's helpful. MiniD is way more powerful than I've thought.
Dec 09 2008
parent reply "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Wed, Dec 10, 2008 at 12:12 AM, davidl <davidl 126.com> wrote:

 I got an impression that a scripting lang coroutine is less expensive than a
function call? I guess coroutine in a scripting interpreted lang won't require
any thread involved, and the suspension of the execution maybe even more
costless compared to a return statement and reexecution of that function in the
case of opApply.


The cost of a coroutine object in MiniD, as well as the cost of
calling it, depends on the compilation options
(http://www.dsource.org/projects/minid/wiki/API2/CompilationOptions).
If you don't specify any version flags when compiling MiniD, it will
use tango Fibers for coroutines created from native functions, and
will use its own system of pausing and resuming for coroutines created
from MiniD functions.

The cost of a coroutine resume/yield is more than a normal function
call, but it's still not a lot.  For example, using MiniD coroutines,
I can perform around 1.6 million resume/yield pairs per second on my
Pentium M.  Fibers aren't much slower; I still get about 1.25 million.
 That's not going to be a bottleneck considering the speed of most
other operations.

Rather, the biggest cost with a coroutine is memory.  A coroutine
object is relatively large compared to a function closure, and
depending on the compilation options, can also allocate D heap memory.
 So for simple iteration tasks, it might be in the interest of
performance to try to write it in the form of a function iterator, and
if it gets to be too complex to be worth it, you can write it as a
coroutine.


 And I'm still not able to get opApply to work with coroutines, any example
available?
 I always get the error message of "Iterated coroutine must be in the initial
state", I've no clue about what it's going on.


When you use foreach on a coroutine, it has to be in the "initial"
state.  The only times a coroutine can be in an initial state is (1)
immediately after they are created, before they are called even once,
and (2) after they have died and then had their .reset() method
called.  A simple example:

function count(x) =
    coroutine function()
    {
        yield() // first "empty" yield is important for foreach-able coroutines
        for(i: 1 .. x + 1)
            yield(i) // yield values after that
        // when this coroutine returns here, iteration stops
    }

foreach(v; count(4))
    writeln(v) // prints 1 through 4

Notice that count is returning a new coroutine object each time you
call it, so the coroutine is always in the "initial" state.

There's more info on coroutines, including showing some iterators and
explaining their various states, at the following link:

http://www.dsource.org/projects/minid/wiki/LanguageSpec2/Functions#Coroutines
Dec 09 2008
parent reply davidl <davidl 126.com> writes:
在 Wed, 10 Dec 2008 13:52:18 +0800，Jarrett Billingsley
<jarrett.billingsley gmail.com> 写道:


 On Wed, Dec 10, 2008 at 12:12 AM, davidl <davidl 126.com> wrote:

 I got an impression that a scripting lang coroutine is less expensive than a
function call? I guess coroutine in a scripting interpreted lang won't require
any thread involved, and the suspension of the execution maybe even more
costless compared to a return statement and reexecution of that function in the
case of opApply.


 The cost of a coroutine object in MiniD, as well as the cost of
 calling it, depends on the compilation options
 (http://www.dsource.org/projects/minid/wiki/API2/CompilationOptions).
 If you don't specify any version flags when compiling MiniD, it will
 use tango Fibers for coroutines created from native functions, and
 will use its own system of pausing and resuming for coroutines created
 from MiniD functions.

 The cost of a coroutine resume/yield is more than a normal function
 call, but it's still not a lot.  For example, using MiniD coroutines,
 I can perform around 1.6 million resume/yield pairs per second on my
 Pentium M.  Fibers aren't much slower; I still get about 1.25 million.
  That's not going to be a bottleneck considering the speed of most
 other operations.

 Rather, the biggest cost with a coroutine is memory.  A coroutine
 object is relatively large compared to a function closure, and
 depending on the compilation options, can also allocate D heap memory.
  So for simple iteration tasks, it might be in the interest of
 performance to try to write it in the form of a function iterator, and
 if it gets to be too complex to be worth it, you can write it as a
 coroutine.


 And I'm still not able to get opApply to work with coroutines, any example
available?
 I always get the error message of "Iterated coroutine must be in the initial
state", I've no clue about what it's going on.


 When you use foreach on a coroutine, it has to be in the "initial"
 state.  The only times a coroutine can be in an initial state is (1)
 immediately after they are created, before they are called even once,
 and (2) after they have died and then had their .reset() method
 called.  A simple example:

 function count(x) =
     coroutine function()
     {
         yield() // first "empty" yield is important for foreach-able coroutines
         for(i: 1 .. x + 1)
             yield(i) // yield values after that
         // when this coroutine returns here, iteration stops
     }

 foreach(v; count(4))
     writeln(v) // prints 1 through 4


It's cool it could be used in that syntax. But I find nowhere of docs
describing this feature?
Do I miss something?

Actually I tried something like:
module test
global v=coroutine function()
{
local opApply=coroutine function opApply(m)
{
local i=0
 if (i<3) {
 i++
 yield(i)
}
return null
}
yield()
}
v()
foreach(m;v)writefln(m)

In the case I wrote, the coroutine foreach just conflicts with the coroutine
foreach way you illustrate
in your example. As I didn't have that information, I tried to perform my
coroutine foreach in a standardized 
opApply way. That's somewhat confusing for people not having the knowledge of
coroutine foreach.
Intuitively, the foreach is presumably working in the way of calling instance's
opApply func and passing the arg
of reverse or not to it. While the truth is I'm wrong. 
 
The inconsistency seems to be a tradeoff. Yes, without opApply it looks clean.
While it's inconsistent. That means
developers who use minid need to learn more. 

The following case illustrates a easily misread code for me at least. In minid,
I thought the parentheses
were compulsive, cause the coroutinefunc.state won't work, but
coroutinefunc.state(). So in such case, I guess the
"count" refers to the function object, while the result shows it just endless
loop and results stack overflow. 
Even that "count" actually a function call without the arg, the calling is
performed. that's somehow misleading.

function count(x) 
{
    function opApply(m) = coroutine function opApply(m)
    {
        for(i: 1 .. x + 1)
            yield(i) // yield values after that
        return null
    }
    foreach(v;count)
        writeln(v) // prints 1 through 4
}
count(4)
object.Exception: Stack Overflow

MDCL stack overflowed out, maybe mdcl should do something to protect itself
from stack overflowing.

I'm trying to use coroutine foreach in a class.

class count 
{
    x =3
    function opApply(m) = coroutine function opApply(m)
    {
        yield()
        for(i: 1 .. m.x + 1)
            yield(i) 
    }

}
global c= count()
    foreach(v;c.opApply(c))
        writeln(v) // prints 1 through 4

it causes of runtime error: Error: opApply(7): Attempting to access field 'x'
from a value of type 'null'

The problem here is I misuseing the function definition. 
function opApply(m) = coroutine function opApply() works OK.

But that's inconsistent from a strong type user's view. Because the left part
is function with 1 arg, right part
is a coroutine function without any arg. That's pretty error-prone for newbies. 

Also the class coroutine foreach tries as follows failed:

class count 
{
    x =3
    function opApply(m) = coroutine function opApply()
    {
        writefln(m) // prints null, that's pretty astonishing for me.
        for(i: 1 .. x + 1)
            yield(i) 
    }

}
global c= count()
    foreach(v;c)
        writeln(v) // prints 1 through 4
the result is printing nothing, and no runtime error.


If I modify the loop to 
    for(i: 1 .. :x + 1)
            yield(i) 
runtime error shows up:
Error: opApply(7): Attempting to access field 'x' from a value of type 'null'

Seems there's no easy way of using coroutine foreach with class instance?


 Notice that count is returning a new coroutine object each time you
 call it, so the coroutine is always in the "initial" state.

 There's more info on coroutines, including showing some iterators and
 explaining their various states, at the following link:

 http://www.dsource.org/projects/minid/wiki/LanguageSpec2/Functions#Coroutines
Dec 10 2008
parent reply "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Wed, Dec 10, 2008 at 3:15 AM, davidl <davidl 126.com> wrote:

 $B:_(B Wed, 10 Dec 2008 13:52:18 +0800$B!$(BJarrett Billingsley
<jarrett.billingsley gmail.com> $B<LF;(B:

 It's cool it could be used in that syntax. But I find nowhere of docs
describing this feature?
 Do I miss something?


Yeah, you do.  I gave you a link in my last post.  See at the end?  Go
there and read all about it.


 Actually I tried something like:
 module test
 global v=coroutine function()
 {
 local opApply=coroutine function opApply(m)
 {
 local i=0
  if (i<3) {
  i++
  yield(i)
 }
 return null
 }
 yield()
 }
 v()
 foreach(m;v)writefln(m)


No wonder it didn't work, it's completely the wrong way to do it.  ;)


 In the case I wrote, the coroutine foreach just conflicts with the coroutine
foreach way you illustrate
 in your example. As I didn't have that information, I tried to perform my
coroutine foreach in a standardized
 opApply way. That's somewhat confusing for people not having the knowledge of
coroutine foreach.
 Intuitively, the foreach is presumably working in the way of calling
instance's opApply func and passing the arg
 of reverse or not to it. While the truth is I'm wrong.


The thing is, it _does_ work by calling the coroutine's opApply.  It's
just that you don't define opApply yourself.  Think about it - you
don't put an opApply in an array to foreach over it.  How can you
opApply over it?  Because array.opApply is defined by the standard
library.  The same thing for coroutines.

http://www.dsource.org/projects/minid/wiki/StdLib2/ThreadLib#Threadmetamethods

That shows where opApply is defined for thread (coroutine) objects.
Which again links to the page I gave you in my last post.


 The inconsistency seems to be a tradeoff. Yes, without opApply it looks clean.
While it's inconsistent. That means
 developers who use minid need to learn more.


Again, it's not inconsistent, as coroutines _do_ have an opApply.


 The following case illustrates a easily misread code for me at least. In
minid, I thought the parentheses
 were compulsive, cause the coroutinefunc.state won't work, but
coroutinefunc.state(). So in such case, I guess the
 "count" refers to the function object, while the result shows it just endless
loop and results stack overflow.
 Even that "count" actually a function call without the arg, the calling is
performed. that's somehow misleading.

 function count(x)
 {
    function opApply(m) = coroutine function opApply(m)
    {
        for(i: 1 .. x + 1)
            yield(i) // yield values after that
        return null
    }
    foreach(v;count)
        writeln(v) // prints 1 through 4
 }
 count(4)
 object.Exception: Stack Overflow


No, it's just that you don't understand MiniD's iteration protocol.
I'm not going to explain it here because I've already explained it -
two or three times - in the documentation.  Long story short, 'count'
refers to the global function 'count' which you already defined.
Functions are first-class values, so "count" refers to the function
object itself, like doing "&count" in D.  Foreach loops are based on
an iterator function, and so when you do foreach(v; count) it thinks
"count" is your iterator function.  It calls it, which makes a
recursive call, which calls itself, and so on.

For information on the iteration protocol, read this:
http://www.dsource.org/projects/minid/wiki/LanguageSpec2/Statements#ForeachStatements


 MDCL stack overflowed out, maybe mdcl should do something to protect itself
from stack overflowing.


Maybe it should ;)


 I'm trying to use coroutine foreach in a class.

 class count
 {
    x =3
    function opApply(m) = coroutine function opApply(m)
    {
        yield()
        for(i: 1 .. m.x + 1)
            yield(i)
    }

 }
 global c= count()
    foreach(v;c.opApply(c))
        writeln(v) // prints 1 through 4

 it causes of runtime error: Error: opApply(7): Attempting to access field 'x'
from a value of type 'null'


Well yeah.  Where is your coroutine getting the value 'm'?  It's not.
So it gets 'null'.


 The problem here is I misuseing the function definition.
 function opApply(m) = coroutine function opApply() works OK.


Yes, because of upvalues.  When you take the parameter off the
coroutine, 'm' then refers to the local variable in the outer
function.


 But that's inconsistent from a strong type user's view. Because the left part
is function with 1 arg, right part
 is a coroutine function without any arg. That's pretty error-prone for newbies.


And coroutines and iterators are pretty advanced.  So I'm not so sure
I much mind what the newbies think ;)

Actually you're again doing it wrong.  You don't have to manually pass
the object to be iterated to opApply, it's already passed as 'this'.
The following:

class count
{
	x = 3

	function opApply()
	{
		local m = this

		return (coroutine function()
		{
			yield()

			for(i: 1 .. m.x + 1)
				yield(i)
		}).opApply() // hee hee
	}
}

global c = count()

foreach(v; c)
	writeln(v) // prints 1 through 4

Works without having to call c.opApply directly.

What I did in the method there is a trick - if you want one opApply to
actually iterate over another object, you can just return the values
from _its_ opApply.  I put 'this' in m so that the coroutine could
access it (because functions can't access their outer function's
'this').  I then create the coroutine function and call opApply on it,
returning those values, so the foreach loop ends up iterating over the
coroutine.


 Also the class coroutine foreach tries as follows failed:

 class count
 {
    x =3
    function opApply(m) = coroutine function opApply()
    {
        writefln(m) // prints null, that's pretty astonishing for me.


Again, where is 'm' coming from?  It's null because you didn't pass
anything.  The object on which opApply is called is in 'this'.


        for(i: 1 .. x + 1)
            yield(i)
    }

 }
 global c= count()
    foreach(v;c)
        writeln(v) // prints 1 through 4
 the result is printing nothing, and no runtime error.


 If I modify the loop to
    for(i: 1 .. :x + 1)
            yield(i)
 runtime error shows up:
 Error: opApply(7): Attempting to access field 'x' from a value of type 'null'


That's because 'this' in the opApply is null.


 Seems there's no easy way of using coroutine foreach with class instance?


No, you're just doing it wrong ;)


 http://www.dsource.org/projects/minid/wiki/LanguageSpec2/Functions#Coroutines




And again, there's the link I gave you last time.  Please read up on it.
Dec 10 2008
parent reply davidl <davidl 126.com> writes:
在 Wed, 10 Dec 2008 21:52:01 +0800，Jarrett Billingsley
<jarrett.billingsley gmail.com> 写道:


 On Wed, Dec 10, 2008 at 3:15 AM, davidl <davidl 126.com> wrote:

 在 Wed, 10 Dec 2008 13:52:18 +0800，Jarrett Billingsley
<jarrett.billingsley gmail.com> 写道:

 It's cool it could be used in that syntax. But I find nowhere of docs
describing this feature?
 Do I miss something?


 Yeah, you do.  I gave you a link in my last post.  See at the end?  Go
 there and read all about it.


Ah, I thought it were another page I've read carefully. Yet that was actually a
page that I read
by skim. I should have read it up carefully. Sorry for that.


 MDCL stack overflowed out, maybe mdcl should do something to protect itself
from stack overflowing.


 Maybe it should ;)


Of course it's your responsibility to improve it. :D And a commercial quality
MiniD interpreter should
 not overflow out. :D


 I'm trying to use coroutine foreach in a class.

 class count
 {
    x =3
    function opApply(m) = coroutine function opApply(m)
    {
        yield()
        for(i: 1 .. m.x + 1)
            yield(i)
    }

 }
 global c= count()
    foreach(v;c.opApply(c))
        writeln(v) // prints 1 through 4

 it causes of runtime error: Error: opApply(7): Attempting to access field 'x'
from a value of type 'null'


 Well yeah.  Where is your coroutine getting the value 'm'?  It's not.
 So it gets 'null'.


 The problem here is I misuseing the function definition.
 function opApply(m) = coroutine function opApply() works OK.


 Yes, because of upvalues.  When you take the parameter off the
 coroutine, 'm' then refers to the local variable in the outer
 function.


 But that's inconsistent from a strong type user's view. Because the left part
is function with 1 arg, right part
 is a coroutine function without any arg. That's pretty error-prone for newbies.


 And coroutines and iterators are pretty advanced.  So I'm not so sure
 I much mind what the newbies think ;)


That's not true. All about compiler is easeing developers. Why not issue a
compiler 
error/warning when it reaches some code like:

function opApply(m) = coroutine function opApply(m) 

1. the local var m shadows the arg m _in 1 declaration_ 
2. this could be a general mistake which could possibly be made by people with 
   a strong type programming background

Probing this kind of code seems troublesome, I will be glad to see MiniD can
give an error
on this kind of code someday. 


 Actually you're again doing it wrong.  You don't have to manually pass
 the object to be iterated to opApply, it's already passed as 'this'.
 The following:

 class count
 {
 	x = 3

 	function opApply()
 	{
 		local m = this

 		return (coroutine function()
 		{
 			yield()

 			for(i: 1 .. m.x + 1)
 				yield(i)
 		}).opApply() // hee hee
 	}
 }

 global c = count()

 foreach(v; c)
 	writeln(v) // prints 1 through 4

 Works without having to call c.opApply directly.

 What I did in the method there is a trick - if you want one opApply to
 actually iterate over another object, you can just return the values
 from _its_ opApply.  I put 'this' in m so that the coroutine could
 access it (because functions can't access their outer function's
 'this').  I then create the coroutine function and call opApply on it,
 returning those values, so the foreach loop ends up iterating over the
 coroutine.


That's pretty tricky. But the syntax of following is cleaner if possible:
class count
{
	x = 3
 
	coroutine function opApply() //mark it as a coroutine function.
	{
	    yield()
 
	    for(i: 1 .. :x + 1)
		yield(i)
	}
}
 
global c = count()
 
foreach(v; c)
	writeln(v) // prints 1 through 4

It's pretty sad "coroutine function" in a whole doesn't work. 
But still opApply() = coroutine function() still not work

You do it: function f()=coroutine function()

I think if MiniD is able to do the syntax I proposed, I wouldn't have those
bunch
questions about foreach on a coroutine function. 
And People won't expect do foreach on a coroutine in most cases if that syntax
comes 
true. Because for most people, a coroutine opApply is more intuitive than a
coroutine 
with an opApply method available.
And I think it's practicle for MiniD compiler decide what to do, because MiniD
can 
probe that if the opApply method is coroutine or not. If it's coroutine
function, than
implicitly a coroutine context created. Also the first yield is quite weird for
me.

class count
{
	x = 3
 
	function opApply() = coroutine function() //mark it as a coroutine function.
	{
	    yield()     // I hope that we could get rid of the first yield, is that
for something special or else?
 
	    for(i: 1 .. :x + 1)
		yield(i)
	}
}
 
global c = count()
 
foreach(v; c)
	writeln(v) // current it prints nothing. I don't know what it's actually going
here.
Dec 10 2008
next sibling parent reply "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Wed, Dec 10, 2008 at 10:21 PM, davidl <davidl 126.com> wrote:

 That's not true. All about compiler is easeing developers. Why not issue a
compiler
 error/warning when it reaches some code like:

 function opApply(m) = coroutine function opApply(m)

 1. the local var m shadows the arg m _in 1 declaration_
 2. this could be a general mistake which could possibly be made by people with
   a strong type programming background

 Probing this kind of code seems troublesome, I will be glad to see MiniD can
give an error
 on this kind of code someday.


I'm not sure that's a good idea.  The compiler does already issue an
error if you declare a local multiple times in the same function:

local x

{
    local x // error, shadows previous declaration
}

But if that is generalized to disallowing local variable declarations
that shadow locals in any enclosing functions as well, .. ech.  It
seems a bit much.  I might implement it, I might not.


 That's pretty tricky. But the syntax of following is cleaner if possible:
 class count
 {
        x = 3

        coroutine function opApply() //mark it as a coroutine function.
        {
            yield()

            for(i: 1 .. :x + 1)
                yield(i)
        }
 }

 global c = count()

 foreach(v; c)
        writeln(v) // prints 1 through 4

 It's pretty sad "coroutine function" in a whole doesn't work.


I think you're confused about what "coroutine" is, grammatically.
"coroutine" is an expression.  Much like "-" in front of "5" gives the
number "-5", "coroutine" in front of an expression that evaluates to a
function gives a thread object.

Allowing something like "coroutine function" as a declaration is
inflexible.  Most of the time, the parameters to the coroutine are not
going to be the same as the parameters to the function which creates
it.  It seems like syntactic noise for little benefit.


 But still opApply() = coroutine function() still not work

 You do it: function f()=coroutine function()


..you don't have a body on that function.  Of course it's not going to work.


 I think if MiniD is able to do the syntax I proposed, I wouldn't have those
bunch
 questions about foreach on a coroutine function.
 And People won't expect do foreach on a coroutine in most cases if that syntax
comes
 true. Because for most people, a coroutine opApply is more intuitive than a
coroutine
 with an opApply method available.


I've considered making coroutines a basic iterable type, in which case
their opApply would disappear.  (it would also make the "dummy" first
index disappear.)


 And I think it's practicle for MiniD compiler decide what to do, because MiniD
can
 probe that if the opApply method is coroutine or not. If it's coroutine
function, than
 implicitly a coroutine context created. Also the first yield is quite weird
for me.


The first yield is there so that any extra parameters to the coroutine
object are passed before iteration starts, as well as to associate the
context ('this') parameter with it.


 class count
 {
        x = 3

        function opApply() = coroutine function() //mark it as a coroutine
function.


Again I think you're getting a bit confused as to what "coroutine" is
doing.  opApply is not a "coroutine function".  opApply is a function
which returns a thread object.  That is,

function opApply() = coroutine function() {}

is equivalent to:

function opApply()
{
    local function blah() {}
    return coroutine blah
}


        {
            yield()     // I hope that we could get rid of the first yield, is
that for something special or else?
            for(i: 1 .. :x + 1)
                yield(i)
        }
 }

 global c = count()

 foreach(v; c)
        writeln(v) // current it prints nothing. I don't know what it's
actually going here.


I'll agree that that's a bit surprising.  What actually happens is
that on the first iteration, it notices that 'c' is not a function, so
it calls c.opApply().  It then happily assumes that opApply returned a
function, and calls it.  That is, it doesn't notice opApply returned a
coroutine, and so doesn't call opApply on the coroutine itself.  It
calls the coroutine, thinking it's a function, and the coroutine
yields nothing, which the foreach loop interprets as "end of
iteration."  Agh.

I'm liking the idea of making 'thread' a basic iterable type more and more ;)
Dec 10 2008
parent reply davidl <davidl 126.com> writes:
在 Thu, 11 Dec 2008 12:45:19 +0800，Jarrett Billingsley
<jarrett.billingsley gmail.com> 写道:


 On Wed, Dec 10, 2008 at 10:21 PM, davidl <davidl 126.com> wrote:

 That's not true. All about compiler is easeing developers. Why not issue a
compiler
 error/warning when it reaches some code like:

 function opApply(m) = coroutine function opApply(m)

 1. the local var m shadows the arg m _in 1 declaration_
 2. this could be a general mistake which could possibly be made by people with
   a strong type programming background

 Probing this kind of code seems troublesome, I will be glad to see MiniD can
give an error
 on this kind of code someday.


 I'm not sure that's a good idea.  The compiler does already issue an
 error if you declare a local multiple times in the same function:

 local x

 {
     local x // error, shadows previous declaration
 }

 But if that is generalized to disallowing local variable declarations
 that shadow locals in any enclosing functions as well, .. ech.  It
 seems a bit much.  I might implement it, I might not.


Umm, I misunderstood the function expression system in MiniD. I didn't expect a
function expression system just thinking that the assignment is just assign a
func
literal to the left hand side as a declaration. 

Because in D you can do a lot coding like following:

void delegate() f= void delegate(){
}

It's obvious for you as the MiniD creator to distinguish the difference. But I
doubt
for normal developer, they could be somewhat confused.

I think a little bit special rule should be added:

function f() = function f(){}   ///disallowed
function f() = function f_exp(){} // allowed

It's a safer rule for people to rename the function with another name, this is 
more clear that for a developer realizing that, he's not doing something like 
I had thought "I assigned a named func literal to the l-value in the
declaration,
which is similar to the D code style". 

And I can hardly see any very bad restriction this could result. And in my
opinion
such function expression literal can even be restricted that it doesn't own a
name. 
Because in such case a name doesn't do any good. For people want to return a
named
literal, they just simply rewrite it to:

function f()
{
   return function f_exp(){}   // allow the named function expressoin literal
here
}

function f() = function f_exp(){} // under stricter rule, f_exp is not allowed.
you write
it:
function f() = function (){}

Please correct me here, if I'm wrong. 

Another thing is I really didn't pay attention to parallel in MiniD. So , the
thread
is not tradition thread in D? It's actually just a coroutine?

I just thought that coroutine was implemented as several different MiniD
interpret 
context for each coroutine, and resume a coroutine just simply resuming that
interpret
context which could be very costless. 

And the thread I though could be some native threads with several MiniD
interpreters working.
Dec 10 2008
parent "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Thu, Dec 11, 2008 at 2:06 AM, davidl <davidl 126.com> wrote:

 Another thing is I really didn't pay attention to parallel in MiniD. So , the
thread
 is not tradition thread in D? It's actually just a coroutine?


Right; it's more like a Fiber from Tango (and is implemented using
Fibers depending on the compilation options).


 I just thought that coroutine was implemented as several different MiniD
interpret
 context for each coroutine, and resume a coroutine just simply resuming that
interpret
 context which could be very costless.


You're actually right, in a way; each thread object is its own
_interpreter_, which has a call stack and locals.  But each thread is
_not_ its own VM; that is, all threads share the same globals and data
can be freely passed between them.

In MiniD, coroutine == thread.  "coroutine" creates a thread object.


 And the thread I though could be some native threads with several MiniD
interpreters working.


MiniD was not designed with symmetric multithreading as a goal.
However, it would still be possible to make a library which you could
use to create and use other VMs from within MiniD.  It's been done
with Lua.
Dec 11 2008
prev sibling parent reply "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Wed, Dec 10, 2008 at 11:45 PM, Jarrett Billingsley
<jarrett.billingsley gmail.com> wrote:

 I think if MiniD is able to do the syntax I proposed, I wouldn't have those
bunch
 questions about foreach on a coroutine function.
 And People won't expect do foreach on a coroutine in most cases if that syntax
comes
 true. Because for most people, a coroutine opApply is more intuitive than a
coroutine
 with an opApply method available.


 I've considered making coroutines a basic iterable type, in which case
 their opApply would disappear.  (it would also make the "dummy" first
 index disappear.)


And they are now :)
Dec 13 2008
parent reply davidl <davidl 126.com> writes:
在 Sat, 13 Dec 2008 23:05:03 +0800，Jarrett Billingsley
<jarrett.billingsley gmail.com> 写道:


 On Wed, Dec 10, 2008 at 11:45 PM, Jarrett Billingsley
 <jarrett.billingsley gmail.com> wrote:

 I think if MiniD is able to do the syntax I proposed, I wouldn't have those
bunch
 questions about foreach on a coroutine function.
 And People won't expect do foreach on a coroutine in most cases if that syntax
comes
 true. Because for most people, a coroutine opApply is more intuitive than a
coroutine
 with an opApply method available.


 I've considered making coroutines a basic iterable type, in which case
 their opApply would disappear.  (it would also make the "dummy" first
 index disappear.)


 And they are now :)
 


Cool, I've learned Scala a bit that I miss several things in scala. 
1.Case class
2.threading

I think those two important features should be considered.

With those two I think it's more sophicated to bind with some GUI interface.

And there , a more active GUI based development could be possible.
Dec 13 2008
next sibling parent "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Sat, Dec 13, 2008 at 11:53 PM, davidl <davidl 126.com> wrote:

 Cool, I've learned Scala a bit that I miss several things in scala.
 1.Case class


I'm not familiar with what a "case class" is, could you demonstrate?


 2.threading


I'm sorry but I don't think SMP will ever make it into the language.
It is open-source, so you're free to modify it and add it if you want.
Dec 13 2008
prev sibling parent "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Sun, Dec 14, 2008 at 12:09 AM, Jarrett Billingsley
<jarrett.billingsley gmail.com> wrote:

 2.threading


 I'm sorry but I don't think SMP will ever make it into the language.
 It is open-source, so you're free to modify it and add it if you want.


Erm, not necessarily SMP, just preemptive multithreading.  Probably
not going to be added, ever.
Dec 13 2008