www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Domain Specific Languages in D; was: C++, D: Dinosaurs?

reply Aarti_pl <aarti interia.pl> writes:
Nick Sabalausky pisze:

 "Piotrek" <starpit tlen.pl> wrote in message
 news:geo14c$ol9$1 digitalmars.com...

 Nick Sabalausky wrote:

 [...]The concept of domain-specific languages is ultra-trendy these 






days

 (probably promoted by the same knuckleheads that hailed things like
 pure-OO, pure-functional, and Extreme Programming as silver 






bullets). But

 I consider domain-specific languages to be purely a symptom of a 






strong

 need for a better general purpose language.


 Totally agree. At my point of view it looks excacly the same. I started
 with pascal then moved to c/c++ at university but I abandoned it 




because

 of its ugliness and non intuitive application development. Then I found
 working with PHP/JAVA much easier but I was disappointed by its
 fundamentals (please let me forget about PHP as a programming 




language).

 And when I was left without any hope I saw a Bright light. :D




(....)


 I agree. I'd much rather just use a sensible API (DBMS-agnostic of 


course).

 Pasting together SQL snippets is just plain ugly, and why dynamically
 generate code just for the DBMS to go and parse it? I mean, geez, 


just cut

 the middleman and pass the request directly. LINQ is better, but I'm 


still

 not convinced it's the right approach. TSQL stored procedures are 


something

 I'd rather, well, just write in a more "normal" language.  And then, of
 course, there's the bizarre "insert" vs. "update" syntax 


inconsistencies and

 general incompatibilities across vendors (ex: insert a row and 


retreive the

 auto-gen id <- which is really a rather common thing to need to do,
 particularly if you've got any foreign keys in the DB design, which 


again is

 rather common).

 To be fair though, I have to say that without some of the advanced 


features




using

 an API for any non-trivial queries could potentially get ugly.


Above subject is very interesting. I didn't know for example about LINQ, 
but have very similar idea to implement Sql queries as specific objects, 
which are later executed.

Currently I implement something similar what is already working, and I 
have to say, it is working very good. My implementation right now is in 
Java (because of easy way to make refactorings with Eclipse), but I want 
to port it into D, and put into my libraries at DSource (Doost project).

For example LINQ statement:
------------------------------
var query1 = from p in people
              where p.Age > 20
              orderby p.Age descending;

------------------------------
will look in my implementation like below:

------------------------------
auto query = Select(people)
		.Where(EqualsOrMore(people.age, 20))
		.OrderBy(people.age, DESC)
		.OrderBy(people.name, ASC);
------------------------------

So it's quite similar to SQL standard, but without drawbacks of building 
SQL's as strings.

------------------------------

As I said before, my implementation is already working. But still I see 
many problems in OO languages (Java, C++, D) which disallows to define 
such sublanguage in a clear way.

Let me enumerate a few:

1. Lack of opImplicitCast()

I see this part as very important and blocking nicer solutions in many 
areas. Hopefully D will get it implemented sooner than later.

For example Where clause in SQL is defined as following:
Where(expression)

expression can be e.g. equality, column or select statement, so there is 
need to put all this cases when defining method "Where". With 
opImplicitCastTo() it would be possible to define it in e.g. Column. 
Then Where will take only objects of Expression. So we get better 
encapsulation.

opImplicitCast would also solve problems with slicing utf8 char arrays, 
as it would be possible to write class/struct with opImplicitCast which 
will convert to char arrays, keeping high level slicing in class/struct.

2. Lack of finer control over operators overloading

Current D implementation makes it impossible to use operators in SQL OO 
sublanguage. Although there is opEquals and opCmp you can not really use 
them as using them you can not distinguish what exactly operation was 
requested and you can not return different type than predefined e.g. 
bool in case of opEquals.

Above have of course its merits, but also disallows to define nice 
looking SQL queries.

To solve this problem I would suggest to allow definition of additional 
operators like below:
opRawEquals
(opRawNotEquals ??)
opRawMore
opRawLess
etc.

then only one: opEquals or opRawEquals can be defined.

It would be also nice to have possibility to define user operators, as 
it seems that it is natural for humans to use some operators in their 
infix form.

3. Impossibility to statically control meta language contructs.

Sql has its own syntax rules and currently they can not be enforced on 
compile time.

E.g. in select statement you would expect to have OrderBy after Where, 
not other way. Currently I can enforce order of clauses only on runtime, 
although it should be probably doable to enforce it on compile time. 
Currently it is also not possible to enforce that e.g. some method must 
be called:

Query query = Select(id).From(table);

Imagine that you want to enforce that in above query From() must be 
always called (Select is an object, From is method on this object).

So what's more is lacking in D, to absorb even more DSL languages?
Thoughts?

PS. If someone is interested in above subject and wants to help in 
porting project from Java and improving it - please contact me on my 
e-mail address.

Regards
Marcin Kuszczak
(aarti_pl)
Nov 04 2008
next sibling parent reply =?iso-8859-1?Q?Julio=20C=e9sar=20Carrascal=20Urquijo?= <jcarrascal gmail.com> writes:
Hello Aarti_pl,


 3. Impossibility to statically control meta language contructs.
 
 Sql has its own syntax rules and currently they can not be enforced on
 compile time.
 
 E.g. in select statement you would expect to have OrderBy after Where,
 not other way.


Instead of returning the full query object you could return a "Fragment" 
query. Something like this:

class WhereFragment
{
    this(Query q) {}
    WhereFragment And(Expression e) {}
    OrderByFragment OrderBy(Column[] columns) {}
}



Currently I can enforce order of clauses only on
 runtime, although it should be probably doable to enforce it on
 compile time. Currently it is also not possible to enforce that e.g.
 some method must be called:

 Query query = Select(id).From(table);
 
 Imagine that you want to enforce that in above query From() must be
 always called (Select is an object, From is method on this object).


You should probably do as LINQ to SQL and start the query with the FROM
statement 
instead. That way you can cascade the type and help auto-complete in IDEs. 
But yes, I don't think it's currently possible to enforce a method to be 
called.
Nov 04 2008
parent reply Aarti_pl <aarti interia.pl> writes:
Julio César Carrascal Urquijo pisze:

 Hello Aarti_pl,
 

 3. Impossibility to statically control meta language contructs.

 Sql has its own syntax rules and currently they can not be enforced on
 compile time.

 E.g. in select statement you would expect to have OrderBy after Where,
 not other way.


 
 Instead of returning the full query object you could return a "Fragment" 
 query. Something like this:
 
 class WhereFragment
 {
    this(Query q) {}
    WhereFragment And(Expression e) {}
    OrderByFragment OrderBy(Column[] columns) {}
 }


Yes, something like this would be possible. But it would start to be 
very complicated, unmanageable and not so useful soon. Implementing it 
in such a way for every DSL will be very hard. Also there are cases in 
normal programs where it would be useful to have such checks.

Currently I use simple runtime checks (somewhat simplified, pseudocodish 
and not tested code below):

class SelectStatement {
   string[] next;
   this() {
     next = ["Where", "From"]; //All possible options after construction
   }

   public SelectStatement Where(SqlExpression exp) {
     if ("Where" !in next) throw Exception("Syntax error");

     ...

     next = ["OrderBy"];
   }
}

Probably it should be possible to implement something similar in 
compiler to make checks on compile time. If some variable has assigned 
object of such a class compiler should trace this variable and check if 
methods are called properly (in order, that they *are* called etc.)

I don't know how easy / difficult is to implement something like this in 
compiler, but I think it may be difficult.


 

 Currently I can enforce order of clauses only on
 runtime, although it should be probably doable to enforce it on
 compile time. Currently it is also not possible to enforce that e.g.
 some method must be called:

 Query query = Select(id).From(table);

 Imagine that you want to enforce that in above query From() must be
 always called (Select is an object, From is method on this object).


 
 You should probably do as LINQ to SQL and start the query with the FROM 
 statement instead. That way you can cascade the type and help 
 auto-complete in IDEs. 


Well, I don't see problem here with my approach... SQL has a lot of 
options after SELECT keyword and before WHERE. How can I use them using 
only From?


 But yes, I don't think it's currently possible to 
 enforce a method to be called.


BR
Marcin Kuszczak
(aarti_pl)
Nov 04 2008
parent reply =?iso-8859-1?Q?Julio=20C=e9sar=20Carrascal=20Urquijo?= <jcarrascal gmail.com> writes:
Hello Aarti_pl,


 Yes, something like this would be possible. But it would start to be
 very complicated, unmanageable and not so useful soon. Implementing it
 in such a way for every DSL will be very hard.


Fragment objects are used in SubSonic so I think it is manageable at least 
for the SQL case.

One thing I should probably have shown in the example is that Fragment objects 
just route calls to the underlying Query object so they are small and don't 
have any logic:

class WhereFragment
{
    this(Query query) { m_query = query; }

    OrderByFragment OrderBy(Column[] columns) 
    { 
        m_query.OrderBy(columns);
    }

    Query m_query;
}

In your case it would be a template of the Table class, of course. Also, 
you would need create a small number of them (WhereFragment, OrderByFragment, 
SelectFragment).



 Also there are cases in
 normal programs where it would be useful to have such checks.


Agree. For example, GUI controls sometimes requires you to set several
properties 
to specific values to get a specific effect. This could benefit from having 
this kind of support from the language. Though I don't think this is a pressing 
issue.



 Currently I use simple runtime checks (somewhat simplified,
 pseudocodish and not tested code below):
 
 class SelectStatement {
 string[] next;
 this() {
 next = ["Where", "From"]; //All possible options after
 construction
 }
 public SelectStatement Where(SqlExpression exp) {
 if ("Where" !in next) throw Exception("Syntax error");
 ...
 
 next = ["OrderBy"];
 }
 }
 Probably it should be possible to implement something similar in
 compiler to make checks on compile time. If some variable has assigned
 object of such a class compiler should trace this variable and check
 if methods are called properly (in order, that they *are* called etc.)


I would say that fragment objects are simpler than littering your methods 
with these checks.




 You should probably do as LINQ to SQL and start the query with the
 FROM statement instead. That way you can cascade the type and help
 auto-complete in IDEs.
 


 Well, I don't see problem here with my approach... SQL has a lot of
 options after SELECT keyword and before WHERE. How can I use them
 using only From?


I meant something like this:

auto q = From(Employee)
    .Where(Employee.Email, Is.EqualTo("jcarrascal gm..."))
    .Select(Employee.ID);

This way the Employee type in the From() would cascade to the Where() and 
the Select() and you could add compile time checks to allow only columns 
of that table to be used in there.

One question: Do you library supports (inner/outer) Joins or have plans to 
support them?

Thanks
Nov 04 2008
parent Marcin Kuszczak <aarti_please_no spam_interia.pl> writes:
Julio César Carrascal Urquijo wrote:


 Hello Aarti_pl,
 

 Yes, something like this would be possible. But it would start to be
 very complicated, unmanageable and not so useful soon. Implementing it
 in such a way for every DSL will be very hard.


 
 Fragment objects are used in SubSonic so I think it is manageable at least
 for the SQL case.


I didn't know about SubSonic. I was thinking that only project which tries to
use OO SQL queries was Ultimate++. So, thanks for reference!


 One thing I should probably have shown in the example is that Fragment
 objects just route calls to the underlying Query object so they are small
 and don't have any logic:
 
 class WhereFragment
 {
     this(Query query) { m_query = query; }
 
     OrderByFragment OrderBy(Column[] columns)
     { 
         m_query.OrderBy(columns);
     }
 
     Query m_query;
 }
 
 In your case it would be a template of the Table class, of course. Also,
 you would need create a small number of them (WhereFragment,
 OrderByFragment, SelectFragment).


This is interesting technique. But also IMHO not always usable as you can not
finish statement with  WhereFragment when your result is SelectStatement. Wait,
I just realized that opImplicitCast will be also very helpful in this case as
WhereFragment can be implicitly casted to Query :-D
 

 Also there are cases in
 normal programs where it would be useful to have such checks.


 
 Agree. For example, GUI controls sometimes requires you to set several
 properties to specific values to get a specific effect. This could benefit
 from having this kind of support from the language. Though I don't think
 this is a pressing issue.
 
 

 Currently I use simple runtime checks (somewhat simplified,
 pseudocodish and not tested code below):
 
 class SelectStatement {
 string[] next;
 this() {
 next = ["Where", "From"]; //All possible options after
 construction
 }
 public SelectStatement Where(SqlExpression exp) {
 if ("Where" !in next) throw Exception("Syntax error");
 ...
 
 next = ["OrderBy"];
 }
 }
 Probably it should be possible to implement something similar in
 compiler to make checks on compile time. If some variable has assigned
 object of such a class compiler should trace this variable and check
 if methods are called properly (in order, that they *are* called etc.)


 
 I would say that fragment objects are simpler than littering your methods
 with these checks.
 


You are probably right here. But I don't have in Java and still in D
opImplicitCast to the rescue and would have to add to fragments artificial e.g.
.apply() method which would always return query.

My design is a bit different from SubSonic one. My queries are quite
independent, and they do not evaluate automatically to results. They can
instead be passed and modified in different places in program. When query is
ready it can be changed into SQL by one of generators. Generator can create SQL
for specific database. But it can also create hash to help caching queries.
After that query is executed by one of available Executor.

So in my design artificial .apply() will not be very nice addition....
 

 You should probably do as LINQ to SQL and start the query with the
 FROM statement instead. That way you can cascade the type and help
 auto-complete in IDEs.
 


 Well, I don't see problem here with my approach... SQL has a lot of
 options after SELECT keyword and before WHERE. How can I use them
 using only From?


 
 I meant something like this:
 
 auto q = From(Employee)
     .Where(Employee.Email, Is.EqualTo("jcarrascal gm..."))
     .Select(Employee.ID);
 
 This way the Employee type in the From() would cascade to the Where() and
 the Select() and you could add compile time checks to allow only columns
 of that table to be used in there.
 


I see. I took a bit different way in this case. In my design From is optional
as all tables are inferred from Select() (every column have defined its table,
so it it easy).


 One question: Do you library supports (inner/outer) Joins or have plans to
 support them?


Yes. I have added them lately.


 Thanks


-- 
Regards
Marcin Kuszczak (Aarti_pl)
-------------------------------------
Ask me why I believe in Jesus - http://www.zapytajmnie.com (en/pl)
Doost (port of few Boost libraries) - http://www.dsource.org/projects/doost/
-------------------------------------
Nov 04 2008
prev sibling parent reply "Nick Sabalausky" <a a.a> writes:
"Aarti_pl" <aarti interia.pl> wrote in message 
news:gepf4o$1118$1 digitalmars.com...

 Nick Sabalausky pisze:

 "Piotrek" <starpit tlen.pl> wrote in message
 news:geo14c$ol9$1 digitalmars.com...

 Nick Sabalausky wrote:

 [...]The concept of domain-specific languages is ultra-trendy these






 days

 (probably promoted by the same knuckleheads that hailed things like
 pure-OO, pure-functional, and Extreme Programming as silver






 bullets). But

 I consider domain-specific languages to be purely a symptom of a






 strong

 need for a better general purpose language.


 Totally agree. At my point of view it looks excacly the same. I started
 with pascal then moved to c/c++ at university but I abandoned it




 because

 of its ugliness and non intuitive application development. Then I found
 working with PHP/JAVA much easier but I was disappointed by its
 fundamentals (please let me forget about PHP as a programming




 language).

 And when I was left without any hope I saw a Bright light. :D




 (....)


 I agree. I'd much rather just use a sensible API (DBMS-agnostic of


 course).

 Pasting together SQL snippets is just plain ugly, and why dynamically
 generate code just for the DBMS to go and parse it? I mean, geez,


 just cut

 the middleman and pass the request directly. LINQ is better, but I'm


 still

 not convinced it's the right approach. TSQL stored procedures are


 something

 I'd rather, well, just write in a more "normal" language.  And then, of
 course, there's the bizarre "insert" vs. "update" syntax


 inconsistencies and

 general incompatibilities across vendors (ex: insert a row and


 retreive the

 auto-gen id <- which is really a rather common thing to need to do,
 particularly if you've got any foreign keys in the DB design, which


 again is

 rather common).

 To be fair though, I have to say that without some of the advanced


 features




 using

 an API for any non-trivial queries could potentially get ugly.


 Above subject is very interesting. I didn't know for example about LINQ, 
 but have very similar idea to implement Sql queries as specific objects, 
 which are later executed.

 Currently I implement something similar what is already working, and I 
 have to say, it is working very good. My implementation right now is in 
 Java (because of easy way to make refactorings with Eclipse), but I want 
 to port it into D, and put into my libraries at DSource (Doost project).

 For example LINQ statement:
 ------------------------------
 var query1 = from p in people
              where p.Age > 20
              orderby p.Age descending;

 ------------------------------
 will look in my implementation like below:

 ------------------------------
 auto query = Select(people)
 .Where(EqualsOrMore(people.age, 20))
 .OrderBy(people.age, DESC)
 .OrderBy(people.name, ASC);
 ------------------------------

 So it's quite similar to SQL standard, but without drawbacks of building 
 SQL's as strings.

 ------------------------------

 As I said before, my implementation is already working. But still I see 
 many problems in OO languages (Java, C++, D) which disallows to define 
 such sublanguage in a clear way.

 Let me enumerate a few:

 1. Lack of opImplicitCast()

 I see this part as very important and blocking nicer solutions in many 
 areas. Hopefully D will get it implemented sooner than later.

 For example Where clause in SQL is defined as following:
 Where(expression)

 expression can be e.g. equality, column or select statement, so there is 
 need to put all this cases when defining method "Where". With 
 opImplicitCastTo() it would be possible to define it in e.g. Column. Then 
 Where will take only objects of Expression. So we get better 
 encapsulation.

 opImplicitCast would also solve problems with slicing utf8 char arrays, as 
 it would be possible to write class/struct with opImplicitCast which will 
 convert to char arrays, keeping high level slicing in class/struct.

 2. Lack of finer control over operators overloading

 Current D implementation makes it impossible to use operators in SQL OO 
 sublanguage. Although there is opEquals and opCmp you can not really use 
 them as using them you can not distinguish what exactly operation was 
 requested and you can not return different type than predefined e.g. bool 
 in case of opEquals.

 Above have of course its merits, but also disallows to define nice looking 
 SQL queries.

 To solve this problem I would suggest to allow definition of additional 
 operators like below:
 opRawEquals
 (opRawNotEquals ??)
 opRawMore
 opRawLess
 etc.

 then only one: opEquals or opRawEquals can be defined.

 It would be also nice to have possibility to define user operators, as it 
 seems that it is natural for humans to use some operators in their infix 
 form.

 3. Impossibility to statically control meta language contructs.

 Sql has its own syntax rules and currently they can not be enforced on 
 compile time.

 E.g. in select statement you would expect to have OrderBy after Where, not 
 other way. Currently I can enforce order of clauses only on runtime, 
 although it should be probably doable to enforce it on compile time. 
 Currently it is also not possible to enforce that e.g. some method must be 
 called:

 Query query = Select(id).From(table);

 Imagine that you want to enforce that in above query From() must be always 
 called (Select is an object, From is method on this object).

 So what's more is lacking in D, to absorb even more DSL languages?
 Thoughts?

 PS. If someone is interested in above subject and wants to help in porting 
 project from Java and improving it - please contact me on my e-mail 
 address.

 Regards
 Marcin Kuszczak
 (aarti_pl)


If we had extension methods, you could change:

.Where(EqualsOrMore(people.age, 20))

to:

.Where(people.age.EqualsOrMore(20))

Which is a little bit nicer. But I agree, a way to return a non-standard 
type from an operator overload would be even better.

Something about both of those above reminds me of a unittesting tool I was 
looking at a while ago...can't remember what it was though...I *think* it 
was a unittesting tool...
Nov 04 2008
next sibling parent Aarti_pl <aarti interia.pl> writes:
Nick Sabalausky pisze:

 But I agree, a way to return a non-standard 
 type from an operator overload would be even better.


Unfortunately it would not be that easy in D.

With current operator overloading method you can not distinguish between 
equals and not equals, as method equals makes comparison itself - it's 
not just notification for user program that some specific operator 
should be called.

As I said before - current behavior makes sense, but it's also problem 
when you want to use &&, ||, == operators in SQL expressions (it 
definitely makes sense in such context and doesn't make user code look 
bad, but is not possible with schema choosen in D).

It would be solved with below:
1. Allow additional operators in D, such as:

T opSymbolEquals(U v1, V v2); //alt. T op==(U v1, V v2);
T opSymbolNotEquals(U v1, V v2); //alt. T op!=(U v1, V v2);
...

for cases where raw operators should be used.

Then it will be possible to use only one standard opEquals() or 
opSymbolEquals in one class.


2. Allow defining user infix operators.

Then it would be possible to define e.g.

opAND()

which is quite good for SQL queries.

BR
Marcin Kuszczak
(aarti_pl)
Nov 05 2008
prev sibling parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Wed, 05 Nov 2008 00:02:36 +0300, Nick Sabalausky <a a.a> wrote:


 Something about both of those above reminds me of a unittesting tool I  
 was
 looking at a while ago...can't remember what it was though...I *think* it
 was a unittesting tool...


NUnit?

  Assert.That( myString, Is.EqualTo("Hello") );
Nov 05 2008
parent "Nick Sabalausky" <a a.a> writes:
"Denis Koroskin" <2korden gmail.com> wrote in message 
news:op.uj48si0oo7cclz proton.creatstudio.intranet...

 On Wed, 05 Nov 2008 00:02:36 +0300, Nick Sabalausky <a a.a> wrote:


 Something about both of those above reminds me of a unittesting tool I 
 was
 looking at a while ago...can't remember what it was though...I *think* it
 was a unittesting tool...


 NUnit?

  Assert.That( myString, Is.EqualTo("Hello") );


Yea, I think that was it.
Nov 05 2008