www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Discussion on groupBy

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
groupBy is an important primitive for relational algebra queries on 
data. Soon to follow are operators such as aggregate() which is a sort 
of reduce() but operating on ranges of ranges. With those in tow, a 
query such as

SELECT COUNT(*), SUM(x) FROM data GROUP BY userid

can be expressed as:

data
   .groupBy!((a, b) => a.userid == b.userid)
   .aggregate(count, (a, b) => a.x + b.x);

We're working the kinks out groupBy now. Those interested please follow 
at https://issues.dlang.org/show_bug.cgi?id=13936.


Thanks,

Andrei
Jan 10 2015
next sibling parent reply "Laeeth Isharc" <laeethnospam spammenot_laeeth.com> writes:
On Saturday, 10 January 2015 at 20:19:14 UTC, Andrei Alexandrescu 
wrote:
 groupBy is an important primitive for relational algebra 
 queries on data. Soon to follow are operators such as 
 aggregate() which is a sort of reduce() but operating on ranges 
 of ranges. With those in tow, a query such as

 SELECT COUNT(*), SUM(x) FROM data GROUP BY userid

 can be expressed as:

 data
   .groupBy!((a, b) => a.userid == b.userid)
   .aggregate(count, (a, b) => a.x + b.x);

 We're working the kinks out groupBy now. Those interested 
 please follow at https://issues.dlang.org/show_bug.cgi?id=13936.
That's great to hear. One factor slowing D adoption might be the difference between being given a complete solution (finished Ikea furniture item) and being handed some raw metal blocks and being told to use the milling machine yourself. The amount of work involves to make the minimal useful solution is actually quite small, but on the one hand the larger part of the benefit accrues to others (which holds some people back), and on the other ability follows a power law, and there are many more script kiddies than Adam Ruppe types in the world. So small frictions have cumulatively large consequences. I think the std.algorithm stuff will come in as very handy as building blocks for an implementation of Pandas like functionality in D. Since it seems that parsing data - both stuctured and unstructured - is in a bull market, this ought to be interesting.
Jan 10 2015
parent reply "Orvid King" <blah38621 gmail.com> writes:
On Saturday, 10 January 2015 at 20:19:14 UTC, Andrei
Alexandrescu wrote:
 groupBy is an important primitive for relational algebra 
 queries on data. Soon to follow are operators such as 
 aggregate() which is a sort of reduce() but operating on ranges 
 of ranges. With those in tow, a query such as

 SELECT COUNT(*), SUM(x) FROM data GROUP BY userid

 can be expressed as:

 data
  .groupBy!((a, b) => a.userid == b.userid)
  .aggregate(count, (a, b) => a.x + b.x);

 We're working the kinks out groupBy now. Those interested 
 please follow at https://issues.dlang.org/show_bug.cgi?id=13936.
It would be interesting if we could make it possible to do a translation between D and SQL, similar to how LINQ is implemented internally, but preferably have it done at compile-time rather than at runtime.
Jan 10 2015
parent "Laeeth Isharc" <laeethnospam nospamlaeeth.com> writes:
 It would be interesting if we could make it possible to do a 
 translation between D and SQL, similar to how LINQ is 
 implemented internally, but preferably have it done at 
 compile-time rather than at runtime.
Have you seen Hibernated? https://github.com/buggins/hibernated
Jan 10 2015
prev sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Saturday, 10 January 2015 at 20:19:14 UTC, Andrei Alexandrescu 
wrote:
 groupBy is an important primitive for relational algebra 
 queries on data. Soon to follow are operators such as 
 aggregate() which is a sort of reduce() but operating on ranges 
 of ranges.
GroupBy is a very important range. It's one the first one ranges I wrote when starting to write D code (being unsatisfied with the existing "group" offering). I agree with separating out the non-equivalence relation groupBy. Having both under the same name just confuses matters and makes for complex implementations.
Jan 11 2015