digitalmars.dip.ideas - Query Expression Sequence (QES)

Richard (Rikki) Andrew Cattermole (214/214) Nov 23 As a design I was inspired from C#'s Linq, it is not tied to a

monkyyy (12/16) Nov 23 Its bad enough that bad languages like python implement regex, at

Richard (Rikki) Andrew Cattermole (8/25) Nov 23 We ship both sqlite and ODBC.

ryuukk_ (6/32) Nov 24 LINQ is not regarded as being a great feature of C#

monkyyy (3/5) Nov 24 Its underengineered, no extensibility, does only one thing,

Paul Backus (4/14) Nov 24 This looks like something straight out of Perl 6/Raku--an entire

Richard (Rikki) Andrew Cattermole <richard cattermole.co.nz> writes:


specific library, it is available based upon the context 
expression.

It should not require AST modifications and can be implemented 
purely in the parser, with lowerings into the library code as per 
the context expression.

I do not believe this should continue, due to a statement by Adam 
Wilson regarding the language integrations of Linq being less 
used today.
However the basic design is described here.

Some examples:

```d
Database db;
Book book;

Person[] authors = book -> Person {
	on: db,
	where: book.authors
};
authors = (book, mydb: db) -> Person {
	on: mydb,
	where book.authors
};

ulong bannedCount = authors -> {
	on: db,
	where: !authors.cool && authors.banned
}.count;
bannedCount = authors -> ulong {
	on: db,
	where: !(authors.cool || authors.profit > 1_000_000) && 
authors.banned,
	result: authors as count
};

Person[] allAuthors = db -> Person {
	from: Person,
	where: Person in -> Person {
		result: Book.authors
	}
};

Person[] allAliveAuthors = db -> Person {
	from: Book.authors as authors,
	where: authors if alive,
	result: authors as unique
};

QESCompiledRef booksForAuthorRef = db -> Book (Person person) {
	from: Book as book,
	where: person in book.authors
}.compile();

auto builder = booksForAuthorRef();
builder["person"] = ...;
Book[] booksForAuthor = builder();
```

Grammar:

```diff
OrOrExpression:
+    QESStart QESExpressionContinue

+ QESExpressionContinue:
+   "->" Type|opt QESExpressionParams|opt '{' QESControlBody '}'

+ QESExpressionParams:
+   QESExpressionParams ',' QESExpressionParams|opt
+   Type Identifier

+ QESStart:
+    Identifier
+    Tuple

+ QESControlBody:
+   QESControlBody ',' QESControlBody|opt
+   Identifier ':' QESControlExpression

+ QESControlExpression:
+   QESControlExpression QESBinaryOp QESControlExpression
+   QESControlExpression "if" QESControlCall
+   QESControlExpression "as" QESControlCall
+   '(' QESControlExpression ')'
+   '-' QESControlExpression
+   '!' QESControlValue
+   QESControlValue

+ QESBinaryOp:
+   "&&"
+   "||"
+   "in"
+   '!' "in"
+   "=="
+   "!="
+   ">="
+   '>'
+   "<="
+   '<'
+   '+'
+   '-'
+   '*'
+   '-'
+   '*'
+   '/'
+   '%'
+   '~'

+ QESControlCall:
+    Identifier '(' QESControlCallArguments ')'
+    Identifier

+ QESControlCallArguments:
+    QESControlCallArguments ',' QESControlCallArguments|opt
+    QESControlExpression

+ QESControlValue:
+    QESExpressionOp
+    IdentifierList
+    QESVariable
+    QESControlLiteral
+    '\' '(' Expression ')'
+    Expression

+ QESVariable:
+    '$' Identifier

+ QESExpressionOp:
+    QESExpression '.' Identifier
+    QESExpression
+    QESExpressionContinue '.' Identifier
+    QESExpressionContinue

+ QESControlLiteral:
+    StringLiteral
+    CharacterLiteral
+    IntegerLiteral
+    FloatLiteral
```

A simplified example of a library type:

```d
struct QESControlState(ResultType, alias Types, alias 
ParameterizedVariableTypes) {
	Types context;
	string[] contextNames, parameterizedVariableNames;

	 disable(__compilerOnly, "User code should not know about the 
control state of a QES")
	static (QESControlRef, QESControlState*) allocate(Types context, 
string[] contextNames, string[] parameterizedVariableNames) {
		QESControlRef ret = QESControlRef.allocate;
		
		ret.state.context = context;
		ret.state.contextNames = contextNames;
		ret.state.parameterizedVariableNames = 
parameterizedVariableNames;

		return (ret, ret.state);
	}

	 disable(__compilerOnly, "User code should not be interacting 
with the parse tree of a QES") {
		// QESControlBody
		ParseTree* control(string identifier);
		// QESControlCall
		ParseTree* queryOp(ResultType, 
Types...)(QESControlState!(ResultType, Types)* other, string 
op=null);
		ParseTree* literal(LiteralType)(LiteralType value);
		ParseTree* expression(ExpressionType)(ExpressionType value);
	}

	static if (is(ResultType == void) {
	} else {
		alias getResult this;
		
		// Your ``getResult`` may return a wrapped slice, i.e. 
``DynamicArray!ResultType``
		ResultType[] getResult();
		
		QESCompiledRef compile();
	}
	
	ulong count();
	bool any();
	bool empty();

	struct QESCompiledRef {
		QESBuilderRef opCall();
	}

	struct QESBuilderRef {
		ParameterizedVariableTypes parameterizedVariables;
		
		void opIndexAssign(PType)(PType value, string 
parameterizedVariable);
		
		static if (is(ResultType == void) {
		} else {
			alias getResult this;
			
			// Your ``getResult`` may return a wrapped slice, i.e. 
``DynamicArray!ResultType``
			ResultType[] getResult();
		}
		
		ulong count();
		bool any();
		bool empty();
	}

	struct ParseTree {
		...
	}
}
```

What the parse tree type should look like:

```d
struct ParseTree {
	ParseTree* negateTruthiness();
	ParseTree* negateNumber();

	ParseTree* asOp(string op);
	ParseTree* checkOp(string op, ParseTree*[] arguments...);
	ParseTree* identifierOp(immutable(string[]) identifiers);
	
	ParseTree* and(ParseTree* rhs);
	ParseTree* or(ParseTree* rhs);
	ParseTree* equals(bool negate, ParseTree* rhs);
	ParseTree* moreThan(bool orEqualsTo, ParseTree* rhs);
	ParseTree* lessThan(bool orEqualsTo, ParseTree* rhs);
	ParseTree* condition(ParseTree* truthy, ParseTree* falsey);
	ParseTree* testInQuery(bool negate, ParseTree* query);
	
	ParseTree* addition(ParseTree* rhs);
	ParseTree* subtract(ParseTree* rhs);
	ParseTree* multiply(ParseTree* rhs);
	ParseTree* divide(ParseTree* rhs);
	ParseTree* modulas(ParseTree* rhs);
	ParseTree* append(ParseTree* rhs);
}
```

Nov 23

monkyyy <crazymonkyyy gmail.com> writes:

On Saturday, 23 November 2024 at 16:44:12 UTC, Richard (Rikki) 
Andrew Cattermole wrote:
 authors = (book, mydb: db) -> Person {
 	on: mydb,
 	where book.authors
 };

Its bad enough that bad languages like python implement regex, at 
least its terse, why would you even want a database language 
embedded in a language that doesnt even ship a database 
datastructure; much less isnt set up to do some multi machine 
number crunching nonsense

If you want database thing, it should at least start with taking 
the preexisting database datastructs and shipping them in phoboes 
and seeing if anyone even uses it

Im not sure about what linq does but Id bet its only on strings 
and isnt extendable in the slightest

Nov 23

"Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:

On 24/11/2024 7:25 AM, monkyyy wrote:
 On Saturday, 23 November 2024 at 16:44:12 UTC, Richard (Rikki) Andrew 
 Cattermole wrote:
 authors = (book, mydb: db) -> Person {
     on: mydb,
     where book.authors
 };

 Its bad enough that bad languages like python implement regex, at least 
 its terse, why would you even want a database language embedded in a 
 language that doesnt even ship a database datastructure; much less isnt 
 set up to do some multi machine number crunching nonsense

We ship both sqlite and ODBC.

So yes, we do offer this currently, although it is not at the level 
people may want.

 If you want database thing, it should at least start with taking the 
 preexisting database datastructs and shipping them in phoboes and seeing 
 if anyone even uses it

I have no offerings for database, that requires coroutines at the bare 
minimum.

 Im not sure about what linq does but Id bet its only on strings and isnt 
 extendable in the slightest

It is basically an ORM with language integrations.

https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/language-specification/expressions#1220-query-expressions

Nov 23

ryuukk_ <ryuukk.dev gmail.com> writes:

On Sunday, 24 November 2024 at 04:00:28 UTC, Richard (Rikki) 
Andrew Cattermole wrote:
 On 24/11/2024 7:25 AM, monkyyy wrote:
 On Saturday, 23 November 2024 at 16:44:12 UTC, Richard (Rikki) 
 Andrew Cattermole wrote:
 authors = (book, mydb: db) -> Person {
     on: mydb,
     where book.authors
 };

 Its bad enough that bad languages like python implement regex, 
 at least its terse, why would you even want a database 
 language embedded in a language that doesnt even ship a 
 database datastructure; much less isnt set up to do some multi 
 machine number crunching nonsense

 We ship both sqlite and ODBC.

 So yes, we do offer this currently, although it is not at the 
 level people may want.

 If you want database thing, it should at least start with 
 taking the preexisting database datastructs and shipping them 
 in phoboes and seeing if anyone even uses it

 I have no offerings for database, that requires coroutines at 
 the bare minimum.

 Im not sure about what linq does but Id bet its only on 
 strings and isnt extendable in the slightest

 It is basically an ORM with language integrations.

 https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/language-specification/expressions#1220-query-expressions



Besides, i have a hard time understanding what your code does, on 
top of being confused about the syntax

This is the epitome of a over engineered feature

Nov 24

monkyyy <crazymonkyyy gmail.com> writes:

On Sunday, 24 November 2024 at 10:55:08 UTC, ryuukk_ wrote:
 
 This is the epitome of a over engineered feature

Its underengineered, no extensibility, does only one thing, 
copies mircosoft

Nov 24

Paul Backus <snarwin gmail.com> writes:

On Saturday, 23 November 2024 at 16:44:12 UTC, Richard (Rikki) 
Andrew Cattermole wrote:

 specific library, it is available based upon the context 
 expression.

 It should not require AST modifications and can be implemented 
 purely in the parser, with lowerings into the library code as 
 per the context expression.

 I do not believe this should continue, due to a statement by 
 Adam Wilson regarding the language integrations of Linq being 
 less used today.
 However the basic design is described here.

This looks like something straight out of Perl 6/Raku--an entire 
DSL built into the language specifically for database queries!

Nov 24

D Programming

C/C++ Programming

Other

digitalmars.dip.ideas - Query Expression Sequence (QES)