www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Linq and the like

reply bearophile <bearophileHUGS lycos.com> writes:
LINQ (and its future parallel extensions), with its additional syntax, may be a
good thing to add to D:
http://www.moserware.com/2008/02/for-loops-using-i-i-enumerators-or-none.html

You can use the same syntax to simplify parallel code, DB access code, etc.

Bye,
bearophile
Feb 03 2008
next sibling parent reply Michiel Helvensteijn <nomail please.com> writes:
bearophile wrote:


 LINQ (and its future parallel extensions), with its additional syntax, may
 be a good thing to add to D:


http://www.moserware.com/2008/02/for-loops-using-i-i-enumerators-or-none.html

 
 You can use the same syntax to simplify parallel code, DB access code,
 etc.


Can't D already do this?

It has lazy evaluation of function literals, so the container class just has
to implement the count method.

Even C++ can do this, though it looks more messy.

Anyway, I disagree with the author about (at least) two things.

* I am one of those people that use the pre-increment operator in for-loops.
I know it doesn't improve performance for integers, but it does for
iterators. So I just use it everywhere for consistency.

* While I agree that when a foreach-loop is possible, you should use it, it
can never completely replace the for-loop, because it is simply less
powerful. Any number of sorting/searching algorithms still need for-loops.

-- 
Michiel
Feb 03 2008
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Michiel Helvensteijn:

 Can't D already do this?


D 2.x has closures, that's a good starting point, but some things are missing
still:
- A built-in support of the methods LINQ accesses in AAs, Arrays, etc.
- Some external bridge, so it can be used on DBMSs too.
- Some sugar for its short syntax, you can see it here:
http://en.wikipedia.org/wiki/Language_Integrated_Query#Language_Extensions
- Multi-processing management for the parallel LINQ.

So as you can see it requires many more things, that's why it's actually useful.



 * While I agree that when a foreach-loop is possible, you should use it, it
 can never completely replace the for-loop, because it is simply less
 powerful. Any number of sorting/searching algorithms still need for-loops.


I agree. And LINQ syntax is slow, so you can't use it in the inner loop of
something that has to run fast. The basic idea is that D can have both
high-level constructs beside keeping the low-level ones.

Bye,
bearophile
Feb 03 2008
parent reply Don Clugston <dac nospam.com.au> writes:
bearophile wrote:

 Michiel Helvensteijn:

 Can't D already do this?


 
 D 2.x has closures, that's a good starting point, but some things are missing
still:
 - A built-in support of the methods LINQ accesses in AAs, Arrays, etc.
 - Some external bridge, so it can be used on DBMSs too.
 - Some sugar for its short syntax, you can see it here:
 http://en.wikipedia.org/wiki/Language_Integrated_Query#Language_Extensions
 - Multi-processing management for the parallel LINQ.
 
 So as you can see it requires many more things, that's why it's actually
useful.
 
 

 * While I agree that when a foreach-loop is possible, you should use it, it
 can never completely replace the for-loop, because it is simply less
 powerful. Any number of sorting/searching algorithms still need for-loops.


 
 I agree. And LINQ syntax is slow, so you can't use it in the inner loop of
something that has to run fast. The basic idea is that D can have both
high-level constructs beside keeping the low-level ones.
 
 Bye,
 bearophile


AFAICT, there's nothing in LINQ that D couldn't (in theory*) do with a library 
-- today implemented using ctfe and mixins, in future with the front end 
replaced by macros.

IMHO, LINQ is a classic example of something that should be in a library. If it 
can't be implemented in a library, then the solution is to make the language 
more powerful, not to shove the library into the compiler.

* not in practice, mainly because of the compiler CTFE bug: temporary strings 
never get deleted, so CTFE runs out of memory extremely quickly.
Feb 04 2008
parent reply "Ellery Newcomer" <ellery-newcomer utulsa.edu> writes:
On Tuesday, 5 February 2008 at 06:28:04 UTC, Don Clugston wrote:

 AFAICT, there's nothing in LINQ that D couldn't (in theory*) do 
 with a library -- today implemented using ctfe and mixins, in 
 future with the front end replaced by macros.

 IMHO, LINQ is a classic example of something that should be in 
 a library. If it can't be implemented in a library, then the 
 solution is to make the language more powerful, not to shove 
 the library into the compiler.

 * not in practice, mainly because of the compiler CTFE bug: 
 temporary strings never get deleted, so CTFE runs out of memory 
 extremely quickly.


LINQ is dependent on .NET's ability to decompose a lambda 
expression into a parse tree, e.g. in converting

qry = qry.Where(a => a.field == localvar);

into an appropriate sql where clause.

What I like about LINQ is it provides a centralized interface for 
high-level query operations that any .NET library worthy of the 
name is going to support. To some extent, ranges and 
std.algorithm, std.range, etc fills this space in D, but it trips 
up in terms of extensibility. A .NET ORM can support the LINQ 
interface; it is an interface and .NET provides the necessary 
tools to implement it (translation of a LINQ query to a SQL 
query).

A D ORM could (and would) support the range interface, but this 
is insufficient for a general query (e.g. you would usually want 
filter!pred(sqlresult) to be done by the database, not by 
std.algorithm). And std.algorithm is an implementation, not an 
interface.


I'm trying to think of how one would implement a rich query 
syntax for a D ORM, and the best I can come up with is something 
like

qry = qry.Where(a => a.eq(localvar));

where a is a proxy type that makes use of operator overloads and 
methods and maybe opDispatch that return query trees. But it's 
ugly. And you can't reuse predicates in ORM queries and phobos 
functions.

Can anyone else think of a better alternative?
Mar 17 2013
parent reply "bls" <nanali orange.fr> writes:
 Can anyone else think of a better alternative?


Compile-time Pegged  (once the CTFE mem issues are solved)

https://github.com/PhilippeSigaud/Pegged
Mar 18 2013
parent "bls" <nanali orange.fr> writes:
On Monday, 18 March 2013 at 10:47:35 UTC, bls wrote:

 Can anyone else think of a better alternative?


 Compile-time Pegged  (once the CTFE mem issues are solved)

 https://github.com/PhilippeSigaud/Pegged


I have to add that in Nemerle Linq is implemented using PEG 
Macros. Same is valid for DBC etc..
Mar 18 2013
prev sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Sunday, 3 February 2008 at 12:25:17 UTC, Michiel Helvensteijn 
wrote:

 bearophile wrote:


 LINQ (and its future parallel extensions), with its additional 
 syntax, may
 be a good thing to add to D:


 http://www.moserware.com/2008/02/for-loops-using-i-i-enumerators-or-none.html

 
 You can use the same syntax to simplify parallel code, DB 
 access code,
 etc.


 Can't D already do this?

 It has lazy evaluation of function literals, so the container 
 class just has
 to implement the count method.

 Even C++ can do this, though it looks more messy.

 Anyway, I disagree with the author about (at least) two things.

 * I am one of those people that use the pre-increment operator 
 in for-loops.
 I know it doesn't improve performance for integers, but it does 
 for
 iterators. So I just use it everywhere for consistency.

 * While I agree that when a foreach-loop is possible, you 
 should use it, it
 can never completely replace the for-loop, because it is simply 
 less
 powerful. Any number of sorting/searching algorithms still need 
 for-loops.


Really?!

for loops can be easily replaced by functional alternatives 
map/reduce.

With tail call optimizations you can even replace them by 
recursive calls without performance loss.

--
Paulo
Mar 18 2013
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/3/2008 1:07 AM, bearophile wrote:

 LINQ (and its future parallel extensions), with its additional syntax, may be
a good thing to add to D:
 http://www.moserware.com/2008/02/for-loops-using-i-i-enumerators-or-none.html

 You can use the same syntax to simplify parallel code, DB access code, etc.


D already has that with ranges and component style programming.
Mar 17 2013
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 17 March 2013 at 21:00:32 UTC, Walter Bright wrote:

 On 2/3/2008 1:07 AM, bearophile wrote:

 LINQ (and its future parallel extensions), with its additional 
 syntax, may be a good thing to add to D:
 http://www.moserware.com/2008/02/for-loops-using-i-i-enumerators-or-none.html

 You can use the same syntax to simplify parallel code, DB 
 access code, etc.


 D already has that with ranges and component style programming.


Yes, but did it have them five years ago? ;)
Mar 17 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/17/2013 2:59 PM, Vladimir Panteleev wrote:

 On Sunday, 17 March 2013 at 21:00:32 UTC, Walter Bright wrote:

 On 2/3/2008 1:07 AM, bearophile wrote:

 LINQ (and its future parallel extensions), with its additional syntax, may be
 a good thing to add to D:
 http://www.moserware.com/2008/02/for-loops-using-i-i-enumerators-or-none.html

 You can use the same syntax to simplify parallel code, DB access code, etc.


 D already has that with ranges and component style programming.


 Yes, but did it have them five years ago? ;)


No. But it doesn't need to be added to D because D has it already.
Mar 17 2013
parent reply "Kapps" <opantm2+spam gmail.com> writes:
On Sunday, 17 March 2013 at 23:55:07 UTC, Walter Bright wrote:

 On 3/17/2013 2:59 PM, Vladimir Panteleev wrote:

 On Sunday, 17 March 2013 at 21:00:32 UTC, Walter Bright wrote:

 On 2/3/2008 1:07 AM, bearophile wrote:

 LINQ (and its future parallel extensions), with its 
 additional syntax, may be
 a good thing to add to D:
 http://www.moserware.com/2008/02/for-loops-using-i-i-enumerators-or-none.html

 You can use the same syntax to simplify parallel code, DB 
 access code, etc.


 D already has that with ranges and component style 
 programming.


 Yes, but did it have them five years ago? ;)


 No. But it doesn't need to be added to D because D has it 
 already.


This thread was posted 5 years ago. :P It just got bumped up 
today.


As for the implementation of lambdas to generate SQL queries and 
the like, there was a pull request for a __traits(codeof) that 
could have been used to do such a thing. Unfortunately, there 
were issues with it in some fundamental way, and it was scrapped. 
A __traits(astof) would be quite interesting for this.
Mar 17 2013
next sibling parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 03/17/2013 05:03 PM, Kapps wrote:

 As for the implementation of lambdas to generate SQL queries and the
 like, there was a pull request for a __traits(codeof) that could have
 been used to do such a thing. Unfortunately, there were issues with it
 in some fundamental way, and it was scrapped. A __traits(astof) would be
 quite interesting for this.


qry = qry.Where(a => a.field == localvar);

Would it be sufficient to get at that local var?
Mar 17 2013
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-03-18 01:03, Kapps wrote:


 This thread was posted 5 years ago. :P It just got bumped up today.


 As for the implementation of lambdas to generate SQL queries and the
 like, there was a pull request for a __traits(codeof) that could have
 been used to do such a thing. Unfortunately, there were issues with it
 in some fundamental way, and it was scrapped. A __traits(astof) would be
 quite interesting for this.


Sounds like AST macro to me.

-- 
/Jacob Carlborg
Mar 18 2013