www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - lodash like utility/algorithms library for D

reply aliak <something something.com> writes:
Hi,

I've been working for fun on a library [0] that is inspired by a 
library from the javascript world called lodash [1]. I basically 
liked the flexibility and thought I'd try and implement a few 
things as it was about the time I started learning D. It 
basically tried to do the same with algorithms as in lodash but 
adds the usage of Optional!T and Expect!(T, U) as ways to returns 
results.

The readme has some more info on how things were approached: 
https://github.com/aliak00/ddash

I've also been inspired by Scala, so stuff from there also got 
thrown in (e.g. Try). I still consider this library largely 
experimental, but it's turning out nice IMO.

Anyway, code:

import std.experimental.all;

void main() {
     import ddash:
         flatMap,
         try_,
         cond,
         compact,
         isFalsey,
         differenceBy;

     // This could throw.
     alias toInt = (string str) => str.to!int;

     // Call toInt using try_ and flatMap so that you only have
     // valid data points
     alias parseDataSet = (arr) => arr.flatMap!(try_!toInt);

     auto cosmicPoints = dataSets
         .map!parseDataSet
          // Get rid of data sets that are empty
         .compact!isFalsey
         // Convert them to CosmicDataPoint type
         .map!(arr => CosmicDataPoint(arr.sum))
         // Remove the known dead-end values by getting a 
difference set
         // based on the "value" member type
         // This can also be:
         // .difference!((a, b) => a.value == b.value)(...)
         // Or in the case of a D value type just:
         // .difference!(...)
         .differenceBy!"value"(knownDeadEndValues);

     assert(cosmicPoints.walkLength == 3);

     // Pattern match on CosmicDataPoint types and process 
accordingly
     alias process = cond!(
         CosmicDataPoint(29), processBlackMatter,
         CosmicDataPoint(30), processGravitationalWave,
         a => a.value > 30 && a.value < 100, processPromisingData,
         a => writeln("This data is useless: ", a),
     );

     cosmicPoints.each!process;
}

// Simulated data sets that result in CosmicDataPoint
immutable dataSets = [
     ["2", "3", "2"],            // Produces CosmicDataPoint(7)
     ["22", "~1", "7"],          // Produces CosmicDataPoint(29)
     ["!$", "88", "3"],          // Produces CosmicDataPoint(91)
     ["junk", "junk", "junk"],   // All junk data points
     ["99", "44"],               // Produces CosmicDataPoint(143)
];

// A cosmic data point
static struct CosmicDataPoint {
     int value;
}

// Simulate values that you know occur but are a dead end
auto knownDeadEndValues = [1, 7, 42].map!(a => 
CosmicDataPoint(a));

// Define some data processing functions
static void processBlackMatter(CosmicDataPoint) {
     writeln("processing black matter discovery");
}
static void processGravitationalWave(CosmicDataPoint) {
     writeln("processing gravitational wave");
}
static void processPromisingData(CosmicDataPoint) {
     writeln("processing data that's close");
}

Cheers,
- Ali

[0] https://ddash.dub.pm
[1] https://lodash.com/
Sep 28 2018
next sibling parent aliak <something something.com> writes:
On Friday, 28 September 2018 at 14:02:48 UTC, aliak wrote:

 Hi,
 [...]


PS Docs: https://aliak00.github.io/ddash/ddash/algorithm.html
Sep 28 2018
prev sibling next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Friday, 28 September 2018 at 14:02:48 UTC, aliak wrote:

 Hi,

 [...]


Lots of good stuff here!

I'm curious about your approach to `Expect`, since I've written a 
version of it myself. How useful have you found being able to use 
unexpected values of any type, as opposed to just exceptions?
Sep 28 2018
parent reply aliak <something something.com> writes:
On Friday, 28 September 2018 at 17:33:04 UTC, Paul Backus wrote:

 On Friday, 28 September 2018 at 14:02:48 UTC, aliak wrote:

 Hi,

 [...]


 Lots of good stuff here!

 I'm curious about your approach to `Expect`, since I've written 
 a version of it myself. How useful have you found being able to 
 use unexpected values of any type, as opposed to just 
 exceptions?


Thanks! Still all rough around the edges.

About Expect, I've found that being able to define a set of 
expected unexpected values is quite practical, and if they're all 
based on the class Exception then there're two problems. 1, it's 
a class so it comes with all those constraints. 2, there's no way 
to close the value domain over the unexpeceted (if that makes 
sense?). Also some functions (take the classic errono catastrophy 
in C) may want to return error codes as unexpected values that 
are ints, and at the same time have a valid value as an int as 
well.

I.e. by allowing you to define the unexepcted you could for 
instance:

enum JSONError {
   invalidKey, notString, notNumber
}

auto a = parse(jsonData);

a.getAsString("key").match!(
     (string value) => // yay
     (JSONError error) => // small domain of what went wrong
);

A bit contrived here, but it actually comes form production code 
(https://github.com/schibsted/account-sdk-ios/blob/master/Source/Core/JSON/JSONObject.swift).

Different language of course, but in that repo there's a type 
called Result, which is basically Expect. But in swift you have 
something called protocols, which lets you contstrain template 
types (in a different way than isInputRange works). And there's a 
standard protocol called Error. So you can do:

Result<T, JSONError> where JSONError is defined as:

struct JSONError: Error {} // conforms to error protocol.

There's actually a D DIP which could allow for similar behavior: 
https://github.com/rikkimax/DIPs/blob/master/DIPs/DIP1xxx-RC.md

But for now since D does not have that.

Another approach would be duck typying I guess. And make a 
isExceptionType trait in D that makes sure some functions are 
supported (i.e. msg, and that it can be constructed with 
__FILE__, __LINE__ ). I have not thought out completely how those 
semantics would work, just thinking out loud right now.
Sep 29 2018
parent reply Paul Backus <snarwin gmail.com> writes:
On Saturday, 29 September 2018 at 12:40:14 UTC, aliak wrote:

 I.e. by allowing you to define the unexepcted you could for 
 instance:

 enum JSONError {
   invalidKey, notString, notNumber
 }

 auto a = parse(jsonData);

 a.getAsString("key").match!(
     (string value) => // yay
     (JSONError error) => // small domain of what went wrong
 );


I agree that this is useful, but why not just return a naked 
`SumType!(string, JSONError)` in that case? Is there some 
additional value added by the `Expect` wrapper that I'm not 
seeing?
Sep 29 2018
parent reply aliak <something something.com> writes:
On Saturday, 29 September 2018 at 19:27:29 UTC, Paul Backus wrote:

 On Saturday, 29 September 2018 at 12:40:14 UTC, aliak wrote:

 I.e. by allowing you to define the unexepcted you could for 
 instance:

 enum JSONError {
   invalidKey, notString, notNumber
 }

 auto a = parse(jsonData);

 a.getAsString("key").match!(
     (string value) => // yay
     (JSONError error) => // small domain of what went wrong
 );


 I agree that this is useful, but why not just return a naked 
 `SumType!(string, JSONError)` in that case? Is there some 
 additional value added by the `Expect` wrapper that I'm not 
 seeing?


That's an option as well I guess. But then you'd need to rely on 
convention and you couldn't do SumType!(int, int) f(), and Expect 
also gives you some more purposeful APIs that makes the code's 
intent clear. Plus I'm considering range behavior as well.

Could you also return a union voldermort type then instead of a 
SumType?
Sep 30 2018
parent reply Paul Backus <snarwin gmail.com> writes:
On Sunday, 30 September 2018 at 22:17:05 UTC, aliak wrote:

 On Saturday, 29 September 2018 at 19:27:29 UTC, Paul Backus 
 wrote:

 I agree that this is useful, but why not just return a naked 
 `SumType!(string, JSONError)` in that case? Is there some 
 additional value added by the `Expect` wrapper that I'm not 
 seeing?


 That's an option as well I guess. But then you'd need to rely 
 on convention and you couldn't do SumType!(int, int) f(), and 
 Expect also gives you some more purposeful APIs that makes the 
 code's intent clear. Plus I'm considering range behavior as 
 well.


Is being able to write `Expect!(int, int)` actually desirable, 
though? It seems to me like being forced to write something like 
`SumType!(int, ErrorCode)` to distinguish the two cases would be 
a good thing, even if ErrorCode itself is just a renamed int 
(e.g., `struct ErrorCode { int code; alias code this; }`).

I guess you could argue that `return 
typeof(return).unexpected(...)` is better than `return 
typeof(return)(ErrorCode(...))`, which is what you'd get with 
SumType, but they look equally ugly to me. What's really needed 
to make that look nice is implicit constructors.

Treating an Expect as a range basically turns it into an 
Optional, in the sense that it collapses any error information it 
contains down to the boolean of empty vs not-empty. In fact, 
probably the easiest way to add range behavior to Expect would be 
to add a method that returns an Optional containing the expected 
value, since Optional already has range behavior.


 Could you also return a union voldermort type then instead of a 
 SumType?


Raw unions in D are horrifically unsafe, so I wouldn't recommend 
it. If you want a voldemort SumType, you can get one like this:

auto f()
{
     struct Result { SumType(T, U) data; alias data this; }
     return Result(...);
}
Sep 30 2018
parent aliak <something something.com> writes:
On Monday, 1 October 2018 at 00:51:24 UTC, Paul Backus wrote:

 On Sunday, 30 September 2018 at 22:17:05 UTC, aliak wrote:

 On Saturday, 29 September 2018 at 19:27:29 UTC, Paul Backus 
 wrote:

 I agree that this is useful, but why not just return a naked 
 `SumType!(string, JSONError)` in that case? Is there some 
 additional value added by the `Expect` wrapper that I'm not 
 seeing?


 That's an option as well I guess. But then you'd need to rely 
 on convention and you couldn't do SumType!(int, int) f(), and 
 Expect also gives you some more purposeful APIs that makes the 
 code's intent clear. Plus I'm considering range behavior as 
 well.


 Is being able to write `Expect!(int, int)` actually desirable, 
 though? It seems to me like being forced to write something 
 like `SumType!(int, ErrorCode)` to distinguish the two cases 
 would be a good thing, even if ErrorCode itself is just a 
 renamed int (e.g., `struct ErrorCode { int code; alias code 
 this; }`).


Hard to say, I would try to avoid it if possible, but why should 
it not be allowed if someone wants it? For now it feels like an 
opinionated restriction that I think is better left out of 
generic code when possible - at least for now. If it turns out 
otherwise I'll change it - this is all still quite experimental 
in me head.

Using SumType to denote success and failure would be more of a 
convention though and would make for more "huh?" moments for 
readability/maintainability, IMO. I like typing intents.

And being able to add an "alias this" in Expect for e.g. might be 
nice. Or if I want to add a "make match has exactly two handlers" 
so that you have to handle both cases would also be a plus.



 I guess you could argue that `return 
 typeof(return).unexpected(...)` is better than `return 
 typeof(return)(ErrorCode(...))`, which is what you'd get with 
 SumType, but they look equally ugly to me. What's really needed 
 to make that look nice is implicit constructors.


*nods*


 Treating an Expect as a range basically turns it into an 
 Optional, in the sense that it collapses any error information 
 it contains down to the boolean of empty vs not-empty. In fact, 
 probably the easiest way to add range behavior to Expect would 
 be to add a method that returns an Optional containing the 
 expected value, since Optional already has range behavior.


Good point. Agreed!


 Could you also return a union voldermort type then instead of 
 a SumType?


 Raw unions in D are horrifically unsafe, so I wouldn't 
 recommend it. If you want a voldemort SumType, you can get one 
 like this:

 auto f()
 {
     struct Result { SumType(T, U) data; alias data this; }
     return Result(...);
 }
Oct 01 2018
prev sibling parent reply Robby Marki <giusoin bobmail.info> writes:
On Friday, 28 September 2018 at 14:02:48 UTC, aliak wrote:

 Hi,

 I've been working for fun on a library [0] that is inspired by 
 a library from the javascript world called lodash [1]. I 
 basically liked the flexibility and thought I'd try and 
 implement a few things as it was about the time I started 
 learning D. It basically tried to do the same with algorithms 
 as in lodash but adds the usage of Optional!T and Expect!(T, U) 
 as ways to returns results.

 [...]


In this example
https://aliak00.github.io/ddash/ddash/functional/try_.html

where does the match function come from?
I get this error when trying to compile:
onlineapp.d(16): Error: no property match for type int
/dlang/dmd/linux/bin64/../../src/phobos/std/algorithm/iteration.d(500):       
instantiated from here: MapResult!(__lambda1, int[])
onlineapp.d(15):        instantiated from here: map!(int[])
Sep 28 2018
parent reply aliak <something something.com> writes:
On Saturday, 29 September 2018 at 01:40:34 UTC, Robby Marki wrote:

 On Friday, 28 September 2018 at 14:02:48 UTC, aliak wrote:

 Hi,

 I've been working for fun on a library [0] that is inspired by 
 a library from the javascript world called lodash [1]. I 
 basically liked the flexibility and thought I'd try and 
 implement a few things as it was about the time I started 
 learning D. It basically tried to do the same with algorithms 
 as in lodash but adds the usage of Optional!T and Expect!(T, 
 U) as ways to returns results.

 [...]


 In this example
 https://aliak00.github.io/ddash/ddash/functional/try_.html

 where does the match function come from?
 I get this error when trying to compile:
 onlineapp.d(16): Error: no property match for type int
 /dlang/dmd/linux/bin64/../../src/phobos/std/algorithm/iteration.d(500):       
instantiated from here: MapResult!(__lambda1, int[])
 onlineapp.d(15):        instantiated from here: map!(int[])


Haha ok I'm a bit stumped right now. That example is wrong, so 
you get the correct error, but it actually compiles on me machine 
right now.

I changed Try.front to return the expected return value of a 
tryable function. And in that example that would clearly be an 
int. So there is no "match" on type int is what I should be 
seeing as well. But for some reason that I'm going to have to dig 
in to, it's compiling a running in the actual project code O_o.
Sep 29 2018
parent aliak <something something.com> writes:
On Saturday, 29 September 2018 at 12:44:38 UTC, aliak wrote:

 On Saturday, 29 September 2018 at 01:40:34 UTC, Robby Marki 
 wrote:

 On Friday, 28 September 2018 at 14:02:48 UTC, aliak wrote:

 [...]


 In this example
 https://aliak00.github.io/ddash/ddash/functional/try_.html

 where does the match function come from?
 I get this error when trying to compile:
 onlineapp.d(16): Error: no property match for type int
 /dlang/dmd/linux/bin64/../../src/phobos/std/algorithm/iteration.d(500):       
instantiated from here: MapResult!(__lambda1, int[])
 onlineapp.d(15):        instantiated from here: map!(int[])


 Haha ok I'm a bit stumped right now. That example is wrong, so 
 you get the correct error, but it actually compiles on me 
 machine right now.

 I changed Try.front to return the expected return value of a 
 tryable function. And in that example that would clearly be an 
 int. So there is no "match" on type int is what I should be 
 seeing as well. But for some reason that I'm going to have to 
 dig in to, it's compiling a running in the actual project code 
 O_o.


Sorry no, it is working as expected. I see that you have an error 
on onlineapp.d(16) ... are you using run.dlang.io?

I just tried a new project and it compiles fine.

Match is part of the Try type.

Cheers,
- Ali
Sep 29 2018