www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - ReQL: pluses and minuses of pipeline-style queries

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Found this on reddit a few days ago: 
http://rob.conery.io/2015/04/17/rethinkdb-2-0-is-amazing/

A good discussion of the pros and cons of pipeline-style queries (the 
ReQL query language reminiscent of D's algorithms/ranges) vs. classic SQL.


Andrei
Apr 24 2015
next sibling parent reply "w0rp" <devw0rp gmail.com> writes:
On Saturday, 25 April 2015 at 05:16:21 UTC, Andrei Alexandrescu 
wrote:

 Found this on reddit a few days ago: 
 http://rob.conery.io/2015/04/17/rethinkdb-2-0-is-amazing/

 A good discussion of the pros and cons of pipeline-style 
 queries (the ReQL query language reminiscent of D's 
 algorithms/ranges) vs. classic SQL.


 Andrei


One thing *kind of* related that I have really enjoyed is 
Django's querysets for building SQL queries. In Django, a 
QuerySet has methods query yields new QuerySet objects, so you 
can build a set of parameters and SQL is eventually generated and 
then executed to yield the results.

andrei_queryset = (
     People.objects
     .filter(first_name="Andrei", last_name__startswith="Al")
     .order_by("-affinity_for_template_metaprogramming")
     .select_related("organisation")
     [0:5]
)

The above would, when evaluated, generate something like the 
following in order to build the objects with.

SELECT p.*, o.* FROM people AS p
INNER JOIN organisation AS o
ON o.id = p.organisation_id
WHERE first_name = "Andrei"
AND last_name LIKE "Al%"
ORDER BY affinity_for_template_metaprogramming DESC
LIMIT 5;

I've been trying to think of a way to create something similar in 
D, maybe for something like HiberateD.
Apr 25 2015
next sibling parent "monty" <monty python.org> writes:
On Saturday, 25 April 2015 at 13:59:41 UTC, w0rp wrote:

 One thing *kind of* related that I have really enjoyed is 
 Django's querysets for building SQL queries. In Django, a 
 QuerySet has methods query yields new QuerySet objects, so you 
 can build a set of parameters and SQL is eventually generated 
 and then executed to yield the results.


another excellent example is ruby's sequel:
http://sequel.jeremyevans.net/
i think its even cleaner and much nicer to work with.
Apr 25 2015
prev sibling parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 26/04/2015 1:59 a.m., w0rp wrote:

 On Saturday, 25 April 2015 at 05:16:21 UTC, Andrei Alexandrescu wrote:

 Found this on reddit a few days ago:
 http://rob.conery.io/2015/04/17/rethinkdb-2-0-is-amazing/

 A good discussion of the pros and cons of pipeline-style queries (the
 ReQL query language reminiscent of D's algorithms/ranges) vs. classic
 SQL.


 Andrei


 One thing *kind of* related that I have really enjoyed is Django's
 querysets for building SQL queries. In Django, a QuerySet has methods
 query yields new QuerySet objects, so you can build a set of parameters
 and SQL is eventually generated and then executed to yield the results.

 andrei_queryset = (
      People.objects
      .filter(first_name="Andrei", last_name__startswith="Al")
      .order_by("-affinity_for_template_metaprogramming")
      .select_related("organisation")
      [0:5]
 )

 The above would, when evaluated, generate something like the following
 in order to build the objects with.

 SELECT p.*, o.* FROM people AS p
 INNER JOIN organisation AS o
 ON o.id = p.organisation_id
 WHERE first_name = "Andrei"
 AND last_name LIKE "Al%"
 ORDER BY affinity_for_template_metaprogramming DESC
 LIMIT 5;

 I've been trying to think of a way to create something similar in D,
 maybe for something like HiberateD.


I'm personally moving towards a DSL.

unittest {
	auto myQuery = """

of table name


from MyModel
where key == $0
as simple



from MyModel
where value contains $0
as complex



""".query;

	MyModel[] gotValues = myQuery.simple("test");
	gotValues = myQuery.perform("complex", "another");
}

Pros:
- Runtime reloadability
- Runtime composability
- More flexible
Cons:
- It's a string
Apr 25 2015
next sibling parent reply "Laeeth Isharc" <nospamlaeeth nospam.laeeth.com> writes:
On Sunday, 26 April 2015 at 01:03:12 UTC, Rikki Cattermole wrote:

 I'm personally moving towards a DSL.

 unittest {
 	auto myQuery = """

 name instead of table name


 from MyModel
 where key == $0
 as simple



 from MyModel
 where value contains $0
 as complex



 """.query;

 	MyModel[] gotValues = myQuery.simple("test");
 	gotValues = myQuery.perform("complex", "another");
 }

 Pros:
 - Runtime reloadability
 - Runtime composability
 - More flexible
 Cons:
 - It's a string


Can Pegged be usefully applied to implement this?

Incidentally, it seems that although Hibernated is very nice, 
there is still work to be done.  Eg I would like to insert 10 
million rows, and it seems like a transaction would be the best 
way of doing so, but it's not yet supported.  No big deal since 
the schema is very simple and I can do it by hand, but it would 
be nice to have at some point.  (I looked at your own ORM, but 
keeping it in memory won't work for me).
Apr 26 2015
parent Rikki Cattermole <alphaglosined gmail.com> writes:
On 27/04/2015 12:10 a.m., Laeeth Isharc wrote:

 On Sunday, 26 April 2015 at 01:03:12 UTC, Rikki Cattermole wrote:

 I'm personally moving towards a DSL.

 unittest {
     auto myQuery = """

 instead of table name


 from MyModel
 where key == $0
 as simple



 from MyModel
 where value contains $0
 as complex



 """.query;

     MyModel[] gotValues = myQuery.simple("test");
     gotValues = myQuery.perform("complex", "another");
 }

 Pros:
 - Runtime reloadability
 - Runtime composability
 - More flexible
 Cons:
 - It's a string


 Can Pegged be usefully applied to implement this?

 Incidentally, it seems that although Hibernated is very nice, there is
 still work to be done.  Eg I would like to insert 10 million rows, and
 it seems like a transaction would be the best way of doing so, but it's
 not yet supported.  No big deal since the schema is very simple and I
 can do it by hand, but it would be nice to have at some point.  (I
 looked at your own ORM, but keeping it in memory won't work for me).


Dvorm like Cmsed is going bye byes.
My next web service framework is going to have a very different approach 
to ORM's and routing!

The given code example, is an actual unittest for the parser. It'll be 
using a reflection API to wrap ORM models up. So the ORM will no longer 
do serialization. Instead the backend will to the appropriate types.

Just so you get a small taste of the reflection capabilities. I know it 
may not seem like much, but imagine e.g. lua to have wrapper code around 
this and to act as if the data models were written natively in it or to 
pass a template (lets say lua based) a bunch of data models and have it 
call D methods on it!
The only problem is that the web server that it is meant to compliment 
isn't ready yet.
https://github.com/DNetDev/webserver

Until I or somebody else gets round to implementing this 
https://github.com/rejectedsoftware/vibe.d/issues/1074 mindset wise, I 
can't continue on.

Also you are welcome to join https://gitter.im/DNetDev/Public if you 
want to talk more.

module webdev.base.reflection.model;
import webdev.base.traits.are : isADataModel, isADataModelProperty, 
isDataModelMemberId, isADataModelQueryMethod, isADataModelQueryStaticMethod;
import webdev.base.traits.have : getDataModelName, 
getDataModelDescription, getDataModelPropertyDescription, 
getDataModelPropertyHints, getDataModelPropertyName;
import webdev.base.udas : OrmPropertyTypes;

private __gshared {
	import std.variant : Algebraic;
	import std.traits : fullyQualifiedName, isArray, ReturnType, 
ParameterTypeTuple;

	AReflectedModel*[string] models;
	AReflectedModel*[string] modelsByTableName;
}

/*
  * Basic interactions of the different kinds of models
  */

/**
  * Gets all names of data models registered
  * Uses the fully qualified name (package + module + class/struct name)
  *
  * Returns:
  * 		The names to all data models
  */
string[] reflectedModelNames() {
	return models.keys;
}

/**
  * Lazily registers and gets a reflected model given the data model type
  *
  * Returns:
  * 		The reflected model
  */
AReflectedModel* getReflectModel(T)() if(isADataModel!T) {
	return getReflectedModel(fullyQualifiedName!T);
}


/**
  * Gets a reflected model based upon its fully qualified name
  *
  * Params:
  * 		name	=	Name of the model
  *
  * Returns:
  * 		The reflected model
  */
AReflectedModel* getReflectedModel(string name) {
	if (name !in models) return null;
	return models[name].dup();
}

/**
  * Gets a reflected model based upon its table name
  *
  * Params:
  * 		name	=	Name of the model
  *
  * Returns:
  * 		The reflected model
  */
AReflectedModel* getReflectedModelByTableName(string name) {
	if (name !in modelsByTableName) return null;
	return modelsByTableName[name].dup();
}

/*
  * General reflection based types
  */

///
enum OrmActualPropertyTypes {
	Unknown,

	UByte,
	Byte,
	UShort,
	Short,
	UInt,
	Int,
	ULong,
	Long,
	Float,
	Double,
	String,
	WString,
	DString,
	
	Array,
	DataModel
}

///
alias ModelValidTypes = Algebraic!(AReflectedModelInstance*, ubyte, 
byte, ushort, short, uint, int, ulong, long, float, double, string, 
wstring, dstring);

///
struct PropertyHint {
	///
	string name;

	///
	OrmActualPropertyTypes actualType;

	///
	OrmActualPropertyTypes arrayActualType;

	///
	AReflectedModel* objectActualType;

	///
	OrmPropertyTypes hintType;

	///
	size_t size;

	///
	string description;

	///
	bool isId;
}

///
struct QueryDescriptor {
	///
	QueryTypeDescriptor[] arguments;

	///
	QueryTypeDescriptor returnType;

	struct QueryTypeDescriptor {
		///
		OrmActualPropertyTypes actualType;
		
		///
		OrmActualPropertyTypes arrayActualType;
		
		///
		AReflectedModel* objectActualType;
	}
}

/*
  * The actual reflection mechanism
  */

struct AReflectedModel {
	static AReflectedModel* reflect(T)() if(isADataModel!T) {
		AReflectedModel* ret = new AReflectedModel;

		ret.dup = () { return AReflectedModel.reflect!T(); };

		/// constructs function delegates for a model instance aware of the type
		void funcCalls(AReflectedModelInstance* retm) {
			static if (__traits(hasMember, T, "isValid")) {
				retm.isValid = () { return (cast(T*)retm.instance_).isValid(); };
			} else {
				retm.isValid = () { return true; };
			}

			retm.get = (string name) {
				foreach(member; __traits(allMembers, T)) {
					static if (isADataModelProperty!(T, member)) {
						if (member == name) {
							return new ModelValidTypes(mixin("(cast(T*)retm.instance_)." ~ 
member));
						}
					}
				}

				return null;
			};

			retm.set = (string name, ModelValidTypes* value) {
				foreach(member; __traits(allMembers, T)) {
					mixin("alias MTYPE = typeof(T." ~ member ~ ");");

					static if (isADataModelProperty!(T, member)) {
						if (member == name) {
							static if (__traits(compiles, {MTYPE t = null;})) {
								if (value is null) {
									mixin("(cast(T*)retm.instance_)." ~ member ~ " = null;");
								} else if (value.convertsTo!MTYPE) {
									mixin("(cast(T*)retm.instance_)." ~ member ~ " = 
value.get!MTYPE;");
								} else {
									assert(0, value.type.toString ~ " is not convertable to " ~ 
MTYPE.stringof);
								}
							} else {
								if (value !is null && value.convertsTo!MTYPE) {
									mixin("(cast(T*)retm.instance_)." ~ member ~ " = 
value.get!MTYPE;");
								} else {
									assert(0, value.type.toString ~ " is not convertable to " ~ 
MTYPE.stringof);
								}
							}
						}
					}
				}
			};

			retm.query = (string name, ModelValidTypes[] values...) {
				foreach(member; __traits(allMembers, T)) {
					mixin("alias MTYPE = typeof(T." ~ member ~ ");");

					static if (isADataModelQueryMethod!(T, member)) {
						alias MRET = ReturnType!MTYPE;

						if (member == name) {
							alias ARGU = ParameterTypeTuple!MTYPE;

							if (values.length != ARGU.length)
								assert(0, "Not enough arguments");
								
							foreach(i, ARG; ARGU) {
								if (!values[i].convertsTo!ARG)
									assert(0, "Wrong types for arguments");
							}

							static if (is(MRET == void)) {
								// return call
								mixin("(cast(T*)retm.instance)." ~ member ~ "(" ~ 
getCallToMethodSyntaxVarient!("values", "ARGU", ARGU) ~ ");");
								return cast(ModelValidTypes[])null;
							} else {
								// call
								mixin("auto ret = (cast(T*)retm.instance_)." ~ member ~ "(" ~ 
getCallToMethodSyntaxVarient!("values", "ARGU", ARGU) ~ ");");

								static if (isArray!MRET) {
									ModelValidTypes[] ret2;

									foreach(v; ret) {
										ret2 ~= ModelValidTypes(v);
									}

									return ret2;
								} else {
									return cast(ModelValidTypes[])[ModelValidTypes(v)];
								}
							}
						}
					}
				}

				assert(0);
			};
		}

		ret.create = () {
			AReflectedModelInstance* retm = new AReflectedModelInstance;
			retm.model_ = ret;

			static if (is(T == class))
				retm.instance_ = &(new T);
			else static if (is(T == struct))
				retm.instance_ = new T;
			else static assert(0);
			
			funcCalls(retm);
			return retm;
		};

		ret.fromInstance = (void* value) {
			/// in
	
			assert(value !is null);

			if (T* ttv = cast(T*)value){}
			else assert(0, "Argument is not of type " ~ T.stringof);

			/// body

			AReflectedModelInstance* retm = new AReflectedModelInstance;
			retm.model_ = ret;

			retm.instance_ = value;

			funcCalls(retm);
			return retm;
		};

		ret.tableName = () { return getDataModelName!T; };
		ret.fullName = () { return fullyQualifiedName!T; };
		ret.description = () { return getDataModelDescription!T; };

		ret.propertyNames = () {
			string[] retm;

			foreach(member; __traits(allMembers, T)) {
				static if (isADataModelProperty!(T, member)) {
					retm ~= member;
				}
			}

			return retm;
		};

		ret.propertyHints = (string name) {
			PropertyHint retm;

			foreach(member; __traits(allMembers, T)) {
				static if (isADataModelProperty!(T, member)) {
					if (member == name) {
						mixin("alias MTYPE = typeof(T." ~ member ~ ");");

						retm.name = getDataModelPropertyName!(T, member);

						// actualType
						enum MTYPEA = actualTypeFromType!MTYPE;
						retm.actualType = MTYPEA;

						// arrayActualType
						static if (MTYPEA == OrmActualPropertyTypes.Array)
							retm.arrayActualType = actualTypeFromType!(typeof(MTYPE.init)[0]);

						// objectActualType
						static if (MTYPEA == OrmActualPropertyTypes.DataModel)
							retm.objectActualType = getReflectModel!MTYPE;

						auto hint = getDataModelPropertyHints!(T, member);

						// hintType
						retm.hintType = hint.type;

						// size
						if (hint.size == 0) {
							static if (isArray!MTYPE) {
							} else
								hint.size = MTYPE.sizeof;
						} else
							retm.size = hint.size;

						// description
						retm.description = getDataModelPropertyDescription!(T, member);

						// isId
						retm.isId = isDataModelMemberId!(T, member);
					}
				}
			}

			return retm;
		};

		ret.query = (string name, ModelValidTypes[] values...) {
			foreach(member; __traits(allMembers, T)) {
				mixin("alias MTYPE = typeof(T." ~ member ~ ");");
				
				static if (isADataModelQueryStaticMethod!(T, member)) {
					alias MRET = ReturnType!MTYPE;
					
					if (member == name) {
						alias ARGU = ParameterTypeTuple!MTYPE;
						
						if (values.length != ARGU.length)
							assert(0, "Not enough arguments");
						
						foreach(i, ARG; ARGU) {
							if (!values[i].convertsTo!ARG)
								assert(0, "Wrong types for arguments");
						}
						
						static if (is(MRET == void)) {
							// return call
							mixin("T." ~ member ~ "(" ~ 
getCallToMethodSyntaxVarient!("values", "ARGU", ARGU) ~ ");");
							return cast(ModelValidTypes[])null;
						} else {
							// call
							mixin("auto ret = T." ~ member ~ "(" ~ 
getCallToMethodSyntaxVarient!("values", "ARGU", ARGU) ~ ");");
							
							ModelValidTypes[] ret2;

							static if (isArray!MRET) {
								foreach(v; ret) {
									static if (is(typeof(v) == class) || is(typeof(v) == struct)) {
										ret2 ~= 
ModelValidTypes(getReflectModel!(typeof(v))().fromInstance(&v));
									} else {
										ret2 ~= ModelValidTypes(v);
									}
								}
							} else {
								static if (is(MRET == class) || is(MRET == struct)) {
									ret2 ~= 
ModelValidTypes(getReflectModel!(MRET)().fromInstance(&ret));
								} else {
									ret2 ~= ModelValidTypes(ret);
								}
							}

							return ret2;
						}
					}
				}
			}
			
			assert(0);
		};

		// queryNames
		ret.queryNames = () {
			string[] ret;

			foreach(member; __traits(allMembers, T)) {
				mixin("alias MTYPE = typeof(T." ~ member ~ ");");
				
				static if (isADataModelQueryStaticMethod!(T, member)) {
					ret ~= member;
				}
			}

			return ret;
		};

		// queryParameters
		ret.queryParameters = (string name) {
			QueryDescriptor ret;
			
			foreach(member; __traits(allMembers, T)) {
				static if (isADataModelQueryStaticMethod!(T, member)) {
					if (member == name) {

						// arguments
						foreach(ARG; ParameterTypeTuple!(mixin("T." ~ name))) {
							QueryDescriptor.QueryTypeDescriptor rett;

							// actualType
							enum MTYPEA = actualTypeFromType!ARG;
							rett.actualType = MTYPEA;

							// arrayActualType
							static if (MTYPEA == OrmActualPropertyTypes.Array)
								rett.arrayActualType = actualTypeFromType!(typeof(ARG.init)[0]);

							// objectActualType
							static if (MTYPEA == OrmActualPropertyTypes.DataModel)
								rett.objectActualType = getReflectModel!ARG;

							ret.arguments ~= rett;
						}

						// return type

						alias RETM = ReturnType!(mixin("T." ~ m));

						// actualType
						enum MTYPEA = actualTypeFromType!RETM;
						ret.returnType.actualType = MTYPEA;
						
						// arrayActualType
						static if (MTYPEA == OrmActualPropertyTypes.Array)
							ret.returnType.arrayActualType = 
actualTypeFromType!(typeof(ARG.init)[0]);
						
						// objectActualType
						static if (MTYPEA == OrmActualPropertyTypes.DataModel)
							ret.returnType.objectActualType = getReflectModel!ARG;
					}
				}
			}
			
			return ret;
		};

		// queryInstanceNames
		ret.queryInstanceNames = () {
			string[] ret;
			
			foreach(member; __traits(allMembers, T)) {
				mixin("alias MTYPE = typeof(T." ~ member ~ ");");
				
				static if (isADataModelQueryMethod!(T, member)) {
					ret ~= member;
				}
			}
			
			return ret;
		};

		// queryInstanceParameters
		ret.queryInstanceParameters = (string name) {
			QueryDescriptor ret;
			
			foreach(member; __traits(allMembers, T)) {
				static if (isADataModelQueryMethod!(T, member)) {
					if (member == name) {
						
						// arguments
						foreach(ARG; ParameterTypeTuple!(mixin("T." ~ name))) {
							QueryDescriptor.QueryTypeDescriptor rett;
							
							// actualType
							enum MTYPEA = actualTypeFromType!ARG;
							rett.actualType = MTYPEA;
							
							// arrayActualType
							static if (MTYPEA == OrmActualPropertyTypes.Array)
								rett.arrayActualType = actualTypeFromType!(typeof(ARG.init)[0]);
							
							// objectActualType
							static if (MTYPEA == OrmActualPropertyTypes.DataModel)
								rett.objectActualType = getReflectModel!ARG;
							
							ret.arguments ~= rett;
						}
						
						// return type
						
						alias RETM = ReturnType!(mixin("T." ~ m));
						
						// actualType
						enum MTYPEA = actualTypeFromType!RETM;
						ret.returnType.actualType = MTYPEA;
						
						// arrayActualType
						static if (MTYPEA == OrmActualPropertyTypes.Array)
							ret.returnType.arrayActualType = 
actualTypeFromType!(typeof(ARG.init)[0]);
						
						// objectActualType
						static if (MTYPEA == OrmActualPropertyTypes.DataModel)
							ret.returnType.objectActualType = getReflectModel!ARG;
					}
				}
			}
			
			return ret;
		};

		models[fullyQualifiedName!T] = ret;
		models[ret.tableName()] = ret;
		return ret;
	}

	AReflectedModel* delegate() dup;
	AReflectedModelInstance* delegate() create;
	AReflectedModelInstance* delegate(void*) fromInstance;

	string delegate() tableName;
	string delegate() fullName;
	string delegate() description;

	const(string[]) delegate() propertyNames;
	PropertyHint delegate(string name) propertyHints;

	ModelValidTypes[] delegate(string func, ModelValidTypes[] values...) query;
	const(string[]) delegate() queryNames;
	QueryDescriptor delegate(string name) queryParameters; //FIXME: should 
not be void*

	const(string[]) delegate() queryInstanceNames;
	QueryDescriptor delegate(string name) queryInstanceParameters; //FIXME: 
should not be void*
}

struct AReflectedModelInstance {
	private {
		AReflectedModel* model_;

		void* instance_;
	}

	AReflectedModel* model() { return model; }
	void* instance() { return instance; }

	/*
	 *
	 * Shouldn't everything below this, stored in the model instead of the 
state?
	 *
	 */

	bool delegate() isValid;

	/*
	 * Get/Set for all approved properties given a name and the value for a 
model instance
	 */
	ModelValidTypes* delegate(string name) get;
	void delegate(string name, ModelValidTypes* value) set;

	ModelValidTypes[] delegate(string func, ModelValidTypes[] values...) query;
}

private {
	OrmActualPropertyTypes actualTypeFromType(T)() {
		static if (is(T == ubyte))
			return OrmActualPropertyTypes.UByte;
		else static if (is(T == byte))
			return OrmActualPropertyTypes.Byte;
		else static if (is(T == ushort))
			return OrmActualPropertyTypes.UShort;
		else static if (is(T == short))
			return OrmActualPropertyTypes.Short;
		else static if (is(T == uint))
			return OrmActualPropertyTypes.UInt;
		else static if (is(T == int))
			return OrmActualPropertyTypes.Int;
		else static if (is(T == ulong))
			return OrmActualPropertyTypes.ULong;
		else static if (is(T == long))
			return OrmActualPropertyTypes.Long;
		else static if (is(T == float))
			return OrmActualPropertyTypes.Float;
		else static if (is(T == double))
			return OrmActualPropertyTypes.Double;
		else static if (is(T == string))
			return OrmActualPropertyTypes.String;
		else static if (is(T == wstring))
			return OrmActualPropertyTypes.WString;
		else static if (is(T == dstring))
			return OrmActualPropertyTypes.DString;
		else static if (is(T == class) || is(T == struct))
			return OrmActualPropertyTypes.DataModel;
		else static if (isArray!T)
			return OrmActualPropertyTypes.Array;
		else
			return OrmActualPropertyTypes.Unknown;
	}

	string getCallToMethodSyntaxVarient(string valuesName, string argName, 
ARGS...)() pure {
		import std.conv : text;
		string ret;

		foreach(i, ARG; ARGS) {
			string TI = text(i);
			ret ~= valuesName ~ "[" ~ TI ~ "].get!(" ~ argName ~ "[" ~ TI ~ "]), ";
		}

		if (ARGS.length > 0)
			ret.length -= 2;

		return ret;
	}
}
Apr 26 2015
prev sibling parent reply "Idan Arye" <GenericNPC gmail.com> writes:
On Sunday, 26 April 2015 at 01:03:12 UTC, Rikki Cattermole wrote:

 I'm personally moving towards a DSL.

 unittest {
 	auto myQuery = """

 name instead of table name


 from MyModel
 where key == $0
 as simple



 from MyModel
 where value contains $0
 as complex



 """.query;

 	MyModel[] gotValues = myQuery.simple("test");
 	gotValues = myQuery.perform("complex", "another");
 }

 Pros:
 - Runtime reloadability
 - Runtime composability
 - More flexible
 Cons:
 - It's a string


I get why people want to use a DSL based on the host language's 
constructs. The host language is built to deal with it's own 
constructs, so composing your queries from them allows you to 
easily do stuff like store query parts in variables, create 
functions that modify queries, use reflection on the queries, 
send variables and expressions from the host process directly to 
the query etc.

What do you gain from a string-based DSL? It doesn't allow you to 
do any of these things - at least not any more 
conveniently/efficiently/safely than you could with SQL strings - 
so essentially it's just a language to replace SQL.

Now, Some languages are so messed up that it's worthwhile to 
create a language the compiles to them(*cough* Javascript 
*cough*). SQL is not one of them. Even if we assume this language 
is better than SQL - is the difference enough to justify leanring 
this new language, to translate SQL errors caused by misuse of 
that langauge(or propagate and let the user deal with it), and 
deal with errors caused by bad implementation of the translation?
Apr 26 2015
parent Rikki Cattermole <alphaglosined gmail.com> writes:
On 27/04/2015 1:12 a.m., Idan Arye wrote:

 On Sunday, 26 April 2015 at 01:03:12 UTC, Rikki Cattermole wrote:

 I'm personally moving towards a DSL.

 unittest {
     auto myQuery = """

 instead of table name


 from MyModel
 where key == $0
 as simple



 from MyModel
 where value contains $0
 as complex



 """.query;

     MyModel[] gotValues = myQuery.simple("test");
     gotValues = myQuery.perform("complex", "another");
 }

 Pros:
 - Runtime reloadability
 - Runtime composability
 - More flexible
 Cons:
 - It's a string


 I get why people want to use a DSL based on the host language's
 constructs. The host language is built to deal with it's own constructs,
 so composing your queries from them allows you to easily do stuff like
 store query parts in variables, create functions that modify queries,
 use reflection on the queries, send variables and expressions from the
 host process directly to the query etc.

 What do you gain from a string-based DSL? It doesn't allow you to do any
 of these things - at least not any more conveniently/efficiently/safely
 than you could with SQL strings - so essentially it's just a language to
 replace SQL.

 Now, Some languages are so messed up that it's worthwhile to create a
 language the compiles to them(*cough* Javascript *cough*). SQL is not
 one of them. Even if we assume this language is better than SQL - is the
 difference enough to justify leanring this new language, to translate
 SQL errors caused by misuse of that langauge(or propagate and let the
 user deal with it), and deal with errors caused by bad implementation of
 the translation?


Worse case scenario, you are free to use the raw sql function for a 
query. But it will only work if its for an SQL database engine.
Also this won't compile to D.
Apr 26 2015
prev sibling parent "Idan Arye" <GenericNPC gmail.com> writes:
On Saturday, 25 April 2015 at 05:16:21 UTC, Andrei Alexandrescu 
wrote:

 Found this on reddit a few days ago: 
 http://rob.conery.io/2015/04/17/rethinkdb-2-0-is-amazing/

 A good discussion of the pros and cons of pipeline-style 
 queries (the ReQL query language reminiscent of D's 
 algorithms/ranges) vs. classic SQL.


 Andrei


I think his example of composability is targeting the exact 
examples where ReQL is composable and SQL isn't. It's easy enough 
to create an opposite example: let's say that our initial query 
is this:

select vendor.name from vendor
inner join catalog on vendor.id = catalog.vendor_id
inner join details on details.catalog_id = catalog.id

r.db("music").table("catalog").map(function(album){
   return {
     artist : album("vendor")("name")
   }
})

If we want to add the `type_id = 2` filter, in SQL it's a matter 
of appending a line:

select vendor.name from vendor
inner join catalog on vendor.id = catalog.vendor_id
inner join details on details.catalog_id = catalog.id
where details.media_type_id = 2;

But in ReQL you'd have to insert the filter in the middle:

r.db("music").table("catalog").filter(function(album){
   return album("details").filter(function(track){
     return track("media_type_id").eq(2)
   })
}).map(function(album){
   return {
     artist : album("vendor")("name")
   }
})

So, composablity seems to be just a matter of where the part you 
want to add happens to be placed in the query's structure...


At this point one might claim that in the SQL case we had to 
modify a string while in the ReQL case we could simply use 
methods of the existing query object - but that's because the 
ReQL query is wrapped for us(the actual format sent to the server 
is not the one written in the source code) while the SQL 
isn't(OK, that's a lie - the SQL query is compiled to some more 
efficient format. But that happens in the driver/server, so the 
bindings don't have to do it). In reality SQL is also wrapped, 
either by entirely new languages(ORMs like Hibernate) or by query 
builders that let you use SQL almost directly, and simply make it 
more convenient(I had good experience with
PetaPoco(http://www.toptensoftware.com/petapoco/), for example). 
If you use these wrappers you get the same composability as 
ReQL(modify query by invoking methods of existing query objects).
Apr 26 2015