www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Random D geekout

reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
I'm writing some code that does some very simplistic parsing, and I'm
just totally geeking out on how awesome D is for writing such code:

	import std.conv;
	import std.regex;
	import std.stdio;

	struct Data {
		string name;
		string phone;
		int age;
		... // a whole bunch of other stuff
	}

	void main() {
		Data d;
		foreach (line; stdin.byLine()) {
			auto m = match(line, "(\w+)\s+(\w+)");
			if (!m) continue;

			auto key = m.captures[1];
			auto value = m.captures[2];

			alias void delegate(string key, string value) attrDg;
			attrDg[string] dgs = [
				"name": delegate(string key, string value) {
					d.name = value;
				},
				"phone": delegate(string key, string value) {
					d.phone = value;
				},
				"age": delegate(string key, string value) {
					d.age = to!int(value);
				},
				...	// whole bunch of other stuff to
					// parse different attributes
			];
			attrDg errordg = delegate(string key, string value) {
				throw Exception("Invalid attribute '%s'"
					.format(key));
			};

			// This is pure awesomeness:
			dgs.get(key.idup, errordg)(key.idup, value.idup);
		}
		// ... do something with Data
	}

Basically, I use std.regex to extract keywords from the input, then use
an AA to map keywords to code that implement said keyword.  That AA of
delegates is just pure awesomeness. AA.get's default value parameter
lets you process keywords and handle errors with a single AA lookup.  I
mean, this is even better than Perl for this kind of text-processing
code!

The only complaint is that I couldn't write auto[string] dgs and have
the compiler auto-infer the delegate type. :-) Additionally, I wasn't
sure if I could omit the "delegate(string,string)" after each keyword;
if that's actually allowed, then this would make D totally pwn Perl!!

(I left out some stuff that makes this code even more of a joy to write:
using nested try/catch blocks, I can throw exceptions from deep-down
parsing code and have the loop that loops over input lines automatically
prefix error messages with the filename/line number where the error
occurred. This way, even errors thrown by to!int() will be formatted
nicely. With Perl, this gets extremely messy due to its pathological use
of $. for line numbers which can get overwritten in unexpected places if
you're processing more than one file at a time.)

Did I mention I'm totally in love with D?? Seriously. It can handle
system-level code and "high-level" text-processing code with total
impunity. What's there not to like?!


T

-- 
Without geometry, life would be pointless. -- VS
Apr 19 2012
next sibling parent "Nick Sabalausky" <SeeWebsiteToContactMe semitwist.com> writes:
"H. S. Teoh" <hsteoh quickfur.ath.cx> wrote in message 
news:mailman.1953.1334894800.4860.digitalmars-d puremagic.com...

 I'm writing some code that does some very simplistic parsing, and I'm
 just totally geeking out on how awesome D is for writing such code:


Heh, yup :)

I grew up on C/C++ (well, after outgrowing BASIC anyway), and one of the 
first things that blew me away about D was its string-processing.


 alias void delegate(string key, string value) attrDg;
 attrDg[string] dgs = [
 "name": delegate(string key, string value) {
 d.name = value;
 },
 "phone": delegate(string key, string value) {
 d.phone = value;
 },
 "age": delegate(string key, string value) {
 d.age = to!int(value);
 },
 ... // whole bunch of other stuff to
 // parse different attributes
 ];


Yea, I've done the same trick :) Fantastic stuff. But the one issue I have 
with it is that you can't do this:

void delegate()[string] dgs = [
    "name": delegate() {
        // do stuff
    },
    "phone": delegate() {
        // do stuff
        dgs["name"](); // ERR! (Shit!)
        // do stuff
    }
];

That limitation is kind of annoying sometimes. I think I filed a ticket for 
it...

http://d.puremagic.com/issues/show_bug.cgi?id=3995

Ahh, shit, it's been marked invalid :(


Did I mention I'm totally in love with D?? Seriously. It can handle
system-level code and "high-level" text-processing code with total
impunity. What's there not to like?!


Yup. Like, totally. :)
Apr 19 2012
prev sibling next sibling parent reply Denis Shelomovskij <verylonglogin.reg gmail.com> writes:
20.04.2012 8:06, H. S. Teoh написал:

 I'm writing some code that does some very simplistic parsing, and I'm
 just totally geeking out on how awesome D is for writing such code:

 	import std.conv;
 	import std.regex;
 	import std.stdio;

 	struct Data {
 		string name;
 		string phone;
 		int age;
 		... // a whole bunch of other stuff
 	}

 	void main() {
 		Data d;
 		foreach (line; stdin.byLine()) {
 			auto m = match(line, "(\w+)\s+(\w+)");


It's better not to create a regex every iteration. Use e.g.
---
auto regEx = regex(`(\w+)\s+(\w+)`);
---
before foreach. Of course, you are not claiming this as a 
high-performance program, but creating a regex every iteration is too 
common mistake to show such code to newbies.


 			if (!m) continue;

 			auto key = m.captures[1];


One `.idup` here will be better. (sorry, just like to nitpick)


 			auto value = m.captures[2];

 			alias void delegate(string key, string value) attrDg;
 			attrDg[string] dgs = [
 				"name": delegate(string key, string value) {
 					d.name = value;
 				},
 				"phone": delegate(string key, string value) {
 					d.phone = value;
 				},
 				"age": delegate(string key, string value) {
 					d.age = to!int(value);
 				},
 				...	// whole bunch of other stuff to
 					// parse different attributes
 			];
 			attrDg errordg = delegate(string key, string value) {
 				throw Exception("Invalid attribute '%s'"
 					.format(key));
 			};

 			// This is pure awesomeness:
 			dgs.get(key.idup, errordg)(key.idup, value.idup);
 		}
 		// ... do something with Data
 	}

 Basically, I use std.regex to extract keywords from the input, then use
 an AA to map keywords to code that implement said keyword.  That AA of
 delegates is just pure awesomeness. AA.get's default value parameter
 lets you process keywords and handle errors with a single AA lookup.  I
 mean, this is even better than Perl for this kind of text-processing
 code!

 The only complaint is that I couldn't write auto[string] dgs and have
 the compiler auto-infer the delegate type. :-) Additionally, I wasn't
 sure if I could omit the "delegate(string,string)" after each keyword;
 if that's actually allowed, then this would make D totally pwn Perl!!


A shorter variant:
---
void delegate(string, string)[string] dgs = [
	"name" : (key, value) { d.name = value; },
	"phone": (key, value) { d.phone = value; },
	"age"  : (key, value) { d.age = to!int(value); },
	...	// whole bunch of other stuff to
		// parse different attributes
];

// `delegate` is needed because otherwise `errordg` will be inferred
// as a `function`, not `delegate` and `dgs.get` will fail
auto errordg = delegate(string key, string value) {
	throw new Exception("Invalid attribute '%s'"
		.format(key));
};
---


 (I left out some stuff that makes this code even more of a joy to write:
 using nested try/catch blocks, I can throw exceptions from deep-down
 parsing code and have the loop that loops over input lines automatically
 prefix error messages with the filename/line number where the error
 occurred. This way, even errors thrown by to!int() will be formatted
 nicely. With Perl, this gets extremely messy due to its pathological use
 of $. for line numbers which can get overwritten in unexpected places if
 you're processing more than one file at a time.)

 Did I mention I'm totally in love with D?? Seriously. It can handle
 system-level code and "high-level" text-processing code with total
 impunity. What's there not to like?!


 T



-- 
Денис В. Шеломовский
Denis V. Shelomovskij
Apr 19 2012
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Apr 20, 2012 at 08:44:06AM +0400, Denis Shelomovskij wrote:

 20.04.2012 8:06, H. S. Teoh написал:

I'm writing some code that does some very simplistic parsing, and I'm
just totally geeking out on how awesome D is for writing such code:

	import std.conv;
	import std.regex;
	import std.stdio;

	struct Data {
		string name;
		string phone;
		int age;
		... // a whole bunch of other stuff
	}

	void main() {
		Data d;
		foreach (line; stdin.byLine()) {
			auto m = match(line, "(\w+)\s+(\w+)");


 
 It's better not to create a regex every iteration. Use e.g.
 ---
 auto regEx = regex(`(\w+)\s+(\w+)`);
 ---
 before foreach. Of course, you are not claiming this as a
 high-performance program, but creating a regex every iteration is
 too common mistake to show such code to newbies.


You're right, it was unoptimized code. I ended up using ctRegex for
them:

	enum attrRx = ctRegex!`...`;
	enum blockRx = ctRegex!`...`;

	if (auto m = match(line, attrRx)) {
		...
	} else if (auto m = match(line, blockRx)) {
		...
	}

The fact that D enums can be arbitrary types is just beyond awesome.


[...]

			auto key = m.captures[1];


 
 One `.idup` here will be better. (sorry, just like to nitpick)


Yeah you're right. I'm refactoring the code right now, and it's much
better to write it this way:

	auto key = m.captures[1].idup;
	auto value = m.captures[2].idup;
	dgs.get(key, invalidAttr)(key, value);

Looks more concise, too.

[...]

 A shorter variant:
 ---
 void delegate(string, string)[string] dgs = [
 	"name" : (key, value) { d.name = value; },
 	"phone": (key, value) { d.phone = value; },
 	"age"  : (key, value) { d.age = to!int(value); },
 	...	// whole bunch of other stuff to
 		// parse different attributes
 ];


[...]

Good idea, I really need to work on my delegate syntax. I must admit I
still have to look it up each time, 'cos I just can't remember the right
syntax with all its shorthands thereof.


T

-- 
Let's not fight disease by killing the patient. -- Sean 'Shaleh' Perry
Apr 19 2012
next sibling parent "Nick Sabalausky" <SeeWebsiteToContactMe semitwist.com> writes:
"H. S. Teoh" <hsteoh quickfur.ath.cx> wrote in message 
news:mailman.1957.1334898572.4860.digitalmars-d puremagic.com...

 Good idea, I really need to work on my delegate syntax. I must admit I
 still have to look it up each time, 'cos I just can't remember the right
 syntax with all its shorthands thereof.


Heh, I can remember the shorthands: It's the full syntax (and syntax for 
referring to the type itself) that I can never remember. One puts "delegate" 
right next to the opening paren, the other puts something else there, meh, I 
can never keep that straight. But the shortcuts I always remember :)
Apr 19 2012
prev sibling parent Ary Manzana <ary esperanto.org.ar> writes:
On 4/20/12 1:09 PM, H. S. Teoh wrote:

 On Fri, Apr 20, 2012 at 08:44:06AM +0400, Denis Shelomovskij wrote:

 20.04.2012 8:06, H. S. Teoh написал:

 I'm writing some code that does some very simplistic parsing, and I'm
 just totally geeking out on how awesome D is for writing such code:

 	import std.conv;
 	import std.regex;
 	import std.stdio;

 	struct Data {
 		string name;
 		string phone;
 		int age;
 		... // a whole bunch of other stuff
 	}

 	void main() {
 		Data d;
 		foreach (line; stdin.byLine()) {
 			auto m = match(line, "(\w+)\s+(\w+)");


 It's better not to create a regex every iteration. Use e.g.
 ---
 auto regEx = regex(`(\w+)\s+(\w+)`);
 ---
 before foreach. Of course, you are not claiming this as a
 high-performance program, but creating a regex every iteration is
 too common mistake to show such code to newbies.


 You're right, it was unoptimized code. I ended up using ctRegex for
 them:

 	enum attrRx = ctRegex!`...`;
 	enum blockRx = ctRegex!`...`;

 	if (auto m = match(line, attrRx)) {
 		...
 	} else if (auto m = match(line, blockRx)) {
 		...
 	}

 The fact that D enums can be arbitrary types is just beyond awesome.


No, enum there means "manifest constant", it has nothing to do with an 
enumeration...
Apr 19 2012
prev sibling next sibling parent reply "F i L" <witte2008 gmail.com> writes:
Denis Shelomovskij wrote:

 A shorter variant:
 ---
 void delegate(string, string)[string] dgs = [
 	"name" : (key, value) { d.name = value; },
 	"phone": (key, value) { d.phone = value; },
 	"age"  : (key, value) { d.age = to!int(value); },
 	...	// whole bunch of other stuff to
 		// parse different attributes
 ];


That's a pretty slick example of D's type inference. This example 
is worthy of a reference in the docs somewhere, IMO. Although, 
written to use UFCS of course:

     auto m = line.match("(\w+)\s+(\w+)");

     ...

     "age" : (key, value) { d.age = value.to!int(); }

:D gotta love UFCS!
Apr 19 2012
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 20.04.2012 9:19, F i L wrote:

 Denis Shelomovskij wrote:

 A shorter variant:
 ---
 void delegate(string, string)[string] dgs = [
 "name" : (key, value) { d.name = value; },
 "phone": (key, value) { d.phone = value; },
 "age" : (key, value) { d.age = to!int(value); },
 ... // whole bunch of other stuff to
 // parse different attributes
 ];




How about putting it on dlang?( [your code here] )

-- 
Dmitry Olshansky
Apr 19 2012
prev sibling next sibling parent "F i L" <witte2008 gmail.com> writes:
Denis Shelomovskij wrote:

 A shorter variant:
 ---
 void delegate(string, string)[string] dgs = [
 	"name" : (key, value) { d.name = value; },
 	"phone": (key, value) { d.phone = value; },
 	"age"  : (key, value) { d.age = to!int(value); },
 	...	// whole bunch of other stuff to
 		// parse different attributes
 ];


That's a pretty slick example of D's type inference. This example 
is worthy of a reference in the docs somewhere, IMO. Although, 
written to use UFCS of course:

     auto m = line.match("(\w+)\s+(\w+)");

     ...

     "age" : (key, value) { d.age = value.to!int(); }

:D gotta love UFCS!
Apr 19 2012
prev sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 20.04.2012 8:44, Denis Shelomovskij wrote:

 20.04.2012 8:06, H. S. Teoh написал:

 I'm writing some code that does some very simplistic parsing, and I'm
 just totally geeking out on how awesome D is for writing such code:

 import std.conv;
 import std.regex;
 import std.stdio;

 struct Data {
 string name;
 string phone;
 int age;
 ... // a whole bunch of other stuff
 }

 void main() {
 Data d;
 foreach (line; stdin.byLine()) {
 auto m = match(line, "(\w+)\s+(\w+)");


 It's better not to create a regex every iteration. Use e.g.
 ---
 auto regEx = regex(`(\w+)\s+(\w+)`);
 ---
 before foreach. Of course, you are not claiming this as a
 high-performance program, but creating a regex every iteration is too
 common mistake to show such code to newbies.


And that's why I pluged this hole - it happens too often. At least up to 
mm... 16 regexes are cached.



-- 
Dmitry Olshansky
Apr 19 2012
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-04-20 06:06, H. S. Teoh wrote:

 I'm writing some code that does some very simplistic parsing, and I'm
 just totally geeking out on how awesome D is for writing such code:

 	import std.conv;
 	import std.regex;
 	import std.stdio;

 	struct Data {
 		string name;
 		string phone;
 		int age;
 		... // a whole bunch of other stuff
 	}

 	void main() {
 		Data d;
 		foreach (line; stdin.byLine()) {
 			auto m = match(line, "(\w+)\s+(\w+)");
 			if (!m) continue;

 			auto key = m.captures[1];
 			auto value = m.captures[2];

 			alias void delegate(string key, string value) attrDg;
 			attrDg[string] dgs = [
 				"name": delegate(string key, string value) {
 					d.name = value;
 				},
 				"phone": delegate(string key, string value) {
 					d.phone = value;
 				},
 				"age": delegate(string key, string value) {
 					d.age = to!int(value);
 				},
 				...	// whole bunch of other stuff to
 					// parse different attributes
 			];
 			attrDg errordg = delegate(string key, string value) {
 				throw Exception("Invalid attribute '%s'"
 					.format(key));
 			};

 			// This is pure awesomeness:
 			dgs.get(key.idup, errordg)(key.idup, value.idup);
 		}
 		// ... do something with Data
 	}

 Basically, I use std.regex to extract keywords from the input, then use
 an AA to map keywords to code that implement said keyword.  That AA of
 delegates is just pure awesomeness. AA.get's default value parameter
 lets you process keywords and handle errors with a single AA lookup.  I
 mean, this is even better than Perl for this kind of text-processing
 code!

 The only complaint is that I couldn't write auto[string] dgs and have
 the compiler auto-infer the delegate type. :-) Additionally, I wasn't
 sure if I could omit the "delegate(string,string)" after each keyword;
 if that's actually allowed, then this would make D totally pwn Perl!!


I think you should be able to write:

"age": (key, value) {
     d.age = to!int(value);
}

Or perhaps even:

"age": (key, value) => d.age = to!int(value);

-- 
/Jacob Carlborg
Apr 20 2012
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 20 Apr 2012 00:06:41 -0400, H. S. Teoh <hsteoh quickfur.ath.cx>  
wrote:



 The only complaint is that I couldn't write auto[string] dgs and have
 the compiler auto-infer the delegate type. :-)


Does this not work?

auto dgs = ...

Also, it doesn't look like that needs to be in the inner loop.  Each time  
you specify an AA literal, it allocates a new one.  So you are allocating  
another AA literal per line.

-Steve
Apr 20 2012
parent reply "Arne" <arne linux.nu> writes:
On Friday, 20 April 2012 at 11:23:49 UTC, Steven Schveighoffer 
wrote:

 On Fri, 20 Apr 2012 00:06:41 -0400, H. S. Teoh 
 <hsteoh quickfur.ath.cx> wrote:



 The only complaint is that I couldn't write auto[string] dgs 
 and have
 the compiler auto-infer the delegate type. :-)


 Does this not work?

 auto dgs = ...

 Also, it doesn't look like that needs to be in the inner loop.  
 Each time you specify an AA literal, it allocates a new one.  
 So you are allocating another AA literal per line.

 -Steve


auto dgs =
[
   "name":  (string value) {d.name = value; },
   "phone": (string value) => cast(void)(d.phone = value),
   "age":   (string value) => cast(void)(d.age   = value.to!int()),
];

This works... is there a better way, to avoid cast?
Apr 20 2012
next sibling parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 04/20/2012 11:50 AM, Arne wrote:

 On Friday, 20 April 2012 at 11:23:49 UTC, Steven Schveighoffer wrote:

 On Fri, 20 Apr 2012 00:06:41 -0400, H. S. Teoh
 <hsteoh quickfur.ath.cx> wrote:



 The only complaint is that I couldn't write auto[string] dgs and have
 the compiler auto-infer the delegate type. :-)


 Does this not work?

 auto dgs = ...

 Also, it doesn't look like that needs to be in the inner loop. Each
 time you specify an AA literal, it allocates a new one. So you are
 allocating another AA literal per line.

 -Steve


 auto dgs =
 [
 "name": (string value) {d.name = value; },
 "phone": (string value) => cast(void)(d.phone = value),
 "age": (string value) => cast(void)(d.age = value.to!int()),
 ];

 This works... is there a better way, to avoid cast?


The => syntax replaces:

- parentheses around the parameter if there is only one parameter
- curly brackets
- the return keyword
- the semicolon at the end of the return statement

   http://dlang.org/expression.html#Lambda

So => is most suitable when there is a single return statement.

Ali
Apr 20 2012
prev sibling next sibling parent reply "Nick Sabalausky" <SeeWebsiteToContactMe semitwist.com> writes:
"Arne" <arne linux.nu> wrote in message 
news:qmehxgyksrdxkabvcyiv forum.dlang.org...

 auto dgs =
 [
   "name":  (string value) {d.name = value; },
   "phone": (string value) => cast(void)(d.phone = value),
   "age":   (string value) => cast(void)(d.age   = value.to!int()),
 ];

 This works... is there a better way, to avoid cast?


Yes: Don't use the lambda syntax when it's not a lambda ;)
Apr 20 2012
parent "Arne" <arne linux.nu> writes:
On Friday, 20 April 2012 at 19:00:29 UTC, Nick Sabalausky wrote:

 "Arne" <arne linux.nu> wrote in message
 news:qmehxgyksrdxkabvcyiv forum.dlang.org...

 auto dgs =
 [
   "name":  (string value) {d.name = value; },
   "phone": (string value) => cast(void)(d.phone = value),
   "age":   (string value) => cast(void)(d.age   = 
 value.to!int()),
 ];

 This works... is there a better way, to avoid cast?


 Yes: Don't use the lambda syntax when it's not a lambda ;)


But... but... that sounds entirely too reasonable! ;)
Apr 21 2012
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-04-20 20:50, Arne wrote:


 auto dgs =
 [
 "name": (string value) {d.name = value; },
 "phone": (string value) => cast(void)(d.phone = value),
 "age": (string value) => cast(void)(d.age = value.to!int()),
 ];

 This works... is there a better way, to avoid cast?


You could try explicitly declare the type of "dgs" and see if that helps.

-- 
/Jacob Carlborg
Apr 21 2012
prev sibling next sibling parent reply "ixid" <nuaccount gmail.com> writes:
As a D learner this thread is very interesting. It would be great 
to maintain it with a polished and error catching version that 
incorporates people's tweaks.
Apr 20 2012
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Apr 20, 2012 at 04:25:30PM +0200, ixid wrote:

 As a D learner this thread is very interesting. It would be great to
 maintain it with a polished and error catching version that
 incorporates people's tweaks.


What I posted was a pared-down simplified version of the actual code I
was working on. I've since done a few more iterations on it, and I
thought I should perhaps post the entire code here as a complete example
of a relatively simple but non-trivial D program that does something
useful.


Some points of note in the code:

- I've factored out the code that does the AA lookups, because I found
  myself needing nested blocks in the input file. So the parseBlock()
  function does the real work, while the input file format is
  essentially specified by the loadSpec() function in the form of a
  nested AA structure containing delegates that implement parsing.  I
  think I like this much better than my original version; it's much more
  reusable, and arguably easier to read.

- Arguably, parseConds can be put into loadSpec() as well, but I'm
  anticipating needing to parse conditions from different places in the
  future, so it's a good idea to put encapsulate it as a separate
  function.

- Exceptions that occur during parsing are caught by code that iterates
  over lines, and exception messages are prefixed with the offending
  filename/line number. This is done in parseBlock(), and it can handle
  exceptions coming from anywhere (such as deeply nested inside one of
  the parsing delegates). Prefixed exceptions are translated into a
  different exception type, so that we don't prefix the same message
  more than once (parseBlock is called reentrantly by some of the
  delegates, so an exception occurring deep inside the call stack may
  traverse several instances of parseBlock during stack unwinding).

- I wrote a simple template for switching between regex() and ctRegex!()
  via a -version flag to the compiler. The regexes have also been
  hoisted out of inner loops, so that it doesn't introduce too much
  unnecessary overhead (besides, they're now compile-time evaluated so
  any such remaining overhead should be minimal). Gotta love D's CTFE
  capabilities!

- Another D coolness: finding the length of the longest name in a People
  array, in a single line of code:

	auto maxnamelen = reduce!"max(a,b.name.length)"(1, spec.people);

- To my great embarrassment, I didn't write any unittests. I should go
  to a corner and weep now.

- I'm sure that there's still lots of room for improvement; I'm by no
  means an expert in D. So criticisms, flames, fanmail, etc., are all
  welcome. ;-)


About the program itself:

The purpose of the program is to automatically assign M people out of a
pool of N people to a recurring event on a rotating basis. For example,
generate a toilet-scrubbing rotation schedule every Saturday for N
roommates, or a guard duty schedule of 2 people each, etc..

While the basic problem is trivial, this program supports conditions and
exclusive tags.

- Conditions basically indicate when a particular person is not
  available (e.g., person is away and won't be back till a certain date,
  person has conflicting event at that time, preplanned vacation,
  preplanned sickness, etc.).

- Exclusive tags basically say that two people having the same tag are
  not to be assigned to the same slot (e.g., they have personality
  conflicts and shouldn't be put together alone, or they're involved
  with another event that can only spare 1 person at a time, etc.).

- There's also a mixup parameter that lets you introduce some randomness
  into the rotation (for example, once in a while pair you up with a
  different person on guard duty, just for some variety).  This is to
  break the tedium of the same old people being assigned together all
  the time.

- Currently the output is just plain ole text. But it's not hard to add
  the capability of outputting, say, iCalendar format, HTML, XML, JSON,
  or whatever your favorite overhyped format is. All you need is to
  appropriately format the contents of the 'assigned' array and the date
  'dt' in genSched().


//----------------------------------snip---------------------------------
/**
 * Simple program to schedule assignments of N people to M slots on a
 * rotational basis.
 */

import std.algorithm;
import std.array;
import std.conv;
import std.datetime;
import std.random;
import std.regex;
import std.stdio;
import std.string;

//version = CtRgx;

template Re(string re) {
	version(CtRgx) {
		enum Re = ctRegex!re;
	} else {
		static Re = regex(re);
	}
}

struct ParseCtxt(R) {
	string fname;
	R      lines;
	int    linenum;
}

class ParseException : Exception {
	this(string msg) {
		super(msg);
	}
}

class Condition {
	abstract bool opCall(Date dt);
}

class SingleDateCond : Condition {
	Date dt;
	this(Date dt) { this.dt = dt; }
}

class DateRangeCond : Condition {
	Date start, end;
	this(Date start, Date end) {
		this.start = start;
		this.end = end;
	}
}

class NotCond : DateRangeCond {
	this(Date single_date) { super(single_date, single_date); }
	this(Date start, Date end) { super(start, end); }
	override bool opCall(Date dt) {
		return dt < this.start || dt > this.end;
	}
}

class AfterCond : SingleDateCond {
	this(Date dt) { super(dt); }
	override bool opCall(Date dt) {
		return dt > this.dt;
	}
}

struct Person {
	string name, phone, email;
	bool[string] tags;
	Condition[] conditions;

	bool eligible(Date dt) {
		foreach (cond; conditions) {
			if (!cond(dt))
				return false;
		}
		return true;
	}
}

struct SchedSpec {
	string   name;
	Date     startdate, enddate;
	Duration period;
	int      ppl_per_event;
	int      mixup;

	Person[] people;

	// People tagged with a tag in this list will not be scheduled together
	bool[string] excl_tags;

	bool[string] getExclTags(Person candidate) {
		bool[string] tags;
		foreach (tag; candidate.tags.keys) {
			if (!(tag in excl_tags))
				continue;

			tags[tag] = true;
		}

		return tags;
	}
}

alias void delegate(string key, string value) attrDg;
alias void delegate(string key) blockDg;

void parseBlock(R)(ref ParseCtxt!R ctxt, attrDg[string] attrDefs,
			blockDg[string] blockDefs=null)
{
	attrDg invalidAttr = delegate(string key, string value) {
		throw new Exception("Unknown attribute '%s'".format(key));
	};
	blockDg invalidBlock = delegate(string key) {
		throw new Exception("Unrecognized block '%s'".format(key));
	};


	auto attrRe  = Re!`^\s*(\w+)\s+(.*)$`;
	auto blockRe = Re!`^\s*(\w+)\s*:\s*$`;
	auto endRe   = Re!`^\s*end\s*$`;

	while (!ctxt.lines.empty) {
		auto line = ctxt.lines.front();
		ctxt.linenum++;

		debug writefln("[%d]>>%s$", ctxt.linenum, line);

		try {
			if (match(line, blankLineRe)) {
				ctxt.lines.popFront();
				continue;	// skip comments & empty lines
			} else if (auto m = match(line, attrRe)) {
				auto key = m.captures[1].idup;
				auto value = m.captures[2].idup;

				attrDefs.get(key, invalidAttr)(key, value);
			} else if (auto m = match(line, blockRe)) {
				auto key = m.captures[1].idup;

				ctxt.lines.popFront();
				blockDefs.get(key, invalidBlock)(key);
			} else if (match(line, endRe)) {
				return;
			} else {
				throw new Exception("Unrecognized spec: %s"
						.format(line));
			}
		} catch(ParseException e) {
			throw e;
		} catch(Exception e) {
			throw new ParseException("%s:%d: %s"
				.format(ctxt.fname, ctxt.linenum, e.msg));
		}

		ctxt.lines.popFront();
	}
}

Date parseDate(string dts) {
	auto dateRe = Re!`^\s*(\d{4})-(\d\d)-(\d\d)\s*$`;
	auto m = match(dts, dateRe);
	if (!m)
		throw new Exception("Invalid date '%s'".format(dts));

	return Date(to!int(m.captures[1]), to!int(m.captures[2]),
		to!int(m.captures[3]));
}

Date[2] parseDateRange(string str) {
	auto rangeRe = Re!`^\s*(\d+-\d+-\d+)(?:\s+to\s+(\d+-\d+-\d+))?`;
	auto m = match(str, rangeRe);
	if (!m)
		throw new Exception("Invalid date range '%s'".format(str));

	auto start = parseDate(m.captures[1]);
	auto end = (m.captures.length >= 2 && m.captures[2].length > 0) ?
			parseDate(m.captures[2]) : start;

	return [start, end];
}

Duration parseDur(string durs) {
	auto daysRe = Re!`^\s*(\d+)\s*days\s*$`;
	auto weeksRe = Re!`^\s*(\d+)\s*weeks\s*$`;

	if (auto m = match(durs, daysRe)) {
		return dur!"days"(to!int(m.captures[1]));
	} else if (auto m = match(durs, weeksRe)) {
		return dur!"weeks"(to!int(m.captures[1]));
	}

	throw new Exception("Unrecognized duration '%s'".format(durs));
}

void parseConds(C)(ref C ctxt, ref Condition[] conds) {
	parseBlock(ctxt, [
		"after": (string key, string value) {
			conds ~= new AfterCond(parseDate(value));
		},
		"not": (string key, string value) {
			auto range = parseDateRange(value);
			conds ~= new NotCond(range[0], range[1]);
		}
	]);
}

SchedSpec loadSpec(string fname) {
	auto specfile = File(fname, "r");
	scope(exit) specfile.close();

	alias typeof(specfile.byLine()) L;
	auto ctxt = ParseCtxt!L(fname, specfile.byLine());

	SchedSpec spec;

	parseBlock(ctxt, [
		"name": (string key, string value) {
			spec.name = value.idup;
		},
		"startdate": (string key, string value) {
			spec.startdate = parseDate(value);
		},
		"enddate": (string key, string value) {
			spec.enddate = parseDate(value);
		},
		"period": (string key, string value) {
			spec.period = parseDur(value);
		},
		"ppl_per_event": (string key, string value) {
			spec.ppl_per_event = to!int(value);
		},
		"mixup": (string key, string value) {
			spec.mixup = to!int(value);
		},
		"exclusive_tags": (string key, string value) {
			foreach (tag; splitter(value, Re!`\s+`)) {
				spec.excl_tags[tag] = true;
			}
		}
	], [
		"person": (string key) {
			Person person;

			parseBlock(ctxt, [
				"name": (string key, string value) {
					person.name = value;
				},
				"phone": (string key, string value) {
					person.phone = value;
				},
				"email": (string key, string value) {
					person.email = value;
				},
				"tags": (string key, string value) {
					foreach (tag; splitter(value,
							Re!`\s+`))
					{
						person.tags[tag] = true;
					}
				}
			], [
				"conditions": (string key) {
					parseConds(ctxt, person.conditions);
				}
			]);

			spec.people ~= person;
		}
	]);

	return spec;
}

void summarize(SchedSpec spec) {
	writefln("Specification '%s':", spec.name);
	writefln("\tStart date: %s", spec.startdate);
	writefln("\tEnd date:   %s", spec.enddate);
	writefln("\tRepeat period:    %s", spec.period);
	writefln("\tPeople per event: %d", spec.ppl_per_event);
	writefln("\tTotal people:     %d", spec.people.length);
}

enum string[] monthAbbrev = [
	"Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct",
	"Nov", "Dec"
];

void genSched(SchedSpec spec) {
	//summarize(spec);

	Person[] queue = spec.people[0..$];
	randomShuffle(queue);

	auto maxnamelen = reduce!"max(a,b.name.length)"(1, spec.people);

	Date dt = spec.startdate;
	while (dt <= spec.enddate) {
		Person[] assigned;
		bool[string] excl_tags;

		foreach (i; 0 .. spec.ppl_per_event) {
			// Find eligible person for assignment
			bool eligible(Person p, Date dt) {
				if (!p.eligible(dt))
					return false;

				// Check for exclusive tag conflicts
				auto ptags = spec.getExclTags(p);
				foreach (tag; ptags.byKey()) {
					if (tag in excl_tags)
						return false;
				}
				return true;
			}
			auto j = countUntil!eligible(queue, dt);

			if (j == -1)
				throw new Exception("No eligible assignees "~
						"found on %s!".format(dt));

			// Move person into assigned list
			assigned ~= queue[j];
			replaceInPlace(queue, j, j+1, cast(Person[])[]);

			// Add person's exclusive tags to the current set of
			// exclusive tags.
			foreach (tag; spec.getExclTags(assigned[$-1]).byKey())
			{
				excl_tags[tag] = true;
			}
		}

		// Put assigned people back to the end of the queue, but with a
		// probability of permuting the order so that we get a mix of
		// different groupings every now and then.
		foreach (p; assigned) {
			insertInPlace(queue, queue.length -
					uniform(0, spec.mixup+1), [p]);
		}

		writef("%04d %s %2d:", dt.year, monthAbbrev[dt.month-1], dt.day);
		foreach (p; assigned) {
			writef("\t%-*s", maxnamelen, p.name);
		}
		writeln();

		dt += spec.period;
	}
}

int showhelp(string progname) {
	writefln("Usage: %s [inputfile]", progname);
	return 1;
}

int main(string[] args) {
	assert(args.length >= 1);
	try {
		if (args.length <= 1)
			return showhelp(args[0]);

		auto inputfile = args[1];
		auto spec = loadSpec(inputfile);
		genSched(spec);
	} catch(Exception e) {
		writefln("Error: %s", e.msg);
	}

	return 0;
}
//----------------------------------snip---------------------------------


T

-- 
Guns don't kill people. Bullets do.
Apr 20 2012
parent reply "SomeDude" <lovelydear mailmetrash.com> writes:
I can't compile it. I get "Out of memory". Is it the regex.d 
module again ?:(
This one really needs to be fixed ASAP, as the older working 
regexp is deprecated.
Apr 21 2012
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 21.04.2012 11:46, SomeDude wrote:

 I can't compile it. I get "Out of memory". Is it the regex.d module
 again ?:(
 This one really needs to be fixed ASAP, as the older working


Ah-ha-ha. OK, come on use it the source are out there in the open :)
It didn't even handle * properly.

  regexp is

 deprecated.


Just stop using ctRegex for now... it's experimental.

Or more to the point the problem is this. I've seen this one on bugzilla:

version(CtRgx) {
		enum Re = ctRegex!re;//auto is OK here BTW
	} else {//that's the problem. It's _parsed_ at compile-time
		static Re = regex(re);//switch static to auto
	}
}

And there is little I can do untill CTFE stops bleeding RAM.

-- 
Dmitry Olshansky
Apr 21 2012
parent reply "SomeDude" <lovelydear mailmetrash.com> writes:
On Saturday, 21 April 2012 at 10:21:49 UTC, Dmitry Olshansky 
wrote:

 Just stop using ctRegex for now... it's experimental.

 Or more to the point the problem is this. I've seen this one on 
 bugzilla:

 version(CtRgx) {
 		enum Re = ctRegex!re;//auto is OK here BTW
 	} else {//that's the problem. It's _parsed_ at compile-time
 		static Re = regex(re);//switch static to auto
 	}
 }

 And there is little I can do untill CTFE stops bleeding RAM.


Well, neither of those works:

  version(CtRgx) {
  	   auto Re = ctRegex!re;//auto is OK here BTW
  	} else {//that's the problem. It's _parsed_ at compile-time
  	   auto Re = regex(re);//switch static to auto
  	}
Apr 21 2012
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 21.04.2012 14:48, SomeDude wrote:

 On Saturday, 21 April 2012 at 10:21:49 UTC, Dmitry Olshansky wrote:

 Just stop using ctRegex for now... it's experimental.

 Or more to the point the problem is this. I've seen this one on bugzilla:

 version(CtRgx) {
 enum Re = ctRegex!re;//auto is OK here BTW
 } else {//that's the problem. It's _parsed_ at compile-time
 static Re = regex(re);//switch static to auto
 }
 }

 And there is little I can do untill CTFE stops bleeding RAM.


 Well, neither of those works:

 version(CtRgx) {
 auto Re = ctRegex!re;//auto is OK here BTW
 } else {//that's the problem. It's _parsed_ at compile-time
 auto Re = regex(re);//switch static to auto
 }


Probably there are other cases where compiler const folds stuff I dunno.

-- 
Dmitry Olshansky
Apr 21 2012
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sat, Apr 21, 2012 at 03:12:05PM +0400, Dmitry Olshansky wrote:

 On 21.04.2012 14:48, SomeDude wrote:

On Saturday, 21 April 2012 at 10:21:49 UTC, Dmitry Olshansky wrote:

Just stop using ctRegex for now... it's experimental.

Or more to the point the problem is this. I've seen this one on bugzilla:

version(CtRgx) {
enum Re = ctRegex!re;//auto is OK here BTW
} else {//that's the problem. It's _parsed_ at compile-time
static Re = regex(re);//switch static to auto
}
}

And there is little I can do untill CTFE stops bleeding RAM.


Well, neither of those works:

version(CtRgx) {
auto Re = ctRegex!re;//auto is OK here BTW
} else {//that's the problem. It's _parsed_ at compile-time
auto Re = regex(re);//switch static to auto
}




[...]

Hmph. I should've checked dmd memory usage when I wrote that. :-(

But anyway, even on my souped up AMD hexacore system, the ctRegex
version takes significantly longer to compile than the non-ctRegex
version. Perhaps I should just avoid ctRegex for now (though it *is* an
ultracool feature of std.regex).

I'm getting confused about the use of 'static' in this context. What I
wanted was to make the regex module-global, but apparently 'static' has
an overloaded meaning here, and also makes it compile-time evaluated?
How do I make it module-global without being compile-time evaluated??


T

-- 
"You know, maybe we don't *need* enemies." "Yeah, best friends are about all I
can take." -- Calvin & Hobbes
Apr 21 2012
next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 21.04.2012 18:41, H. S. Teoh wrote:
[snip]

 How do I make it module-global without being compile-time evaluated??


No idea ;)

But as a workaround:

Global blah;

static this(){
	blah = ...;
}

-- 
Dmitry Olshansky
Apr 21 2012
prev sibling next sibling parent "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Saturday, 21 April 2012 at 14:40:00 UTC, H. S. Teoh wrote:

 I'm getting confused about the use of 'static' in this context. 
 What I
 wanted was to make the regex module-global, but apparently 
 'static' has
 an overloaded meaning here, and also makes it compile-time 
 evaluated?
 How do I make it module-global without being compile-time 
 evaluated??


 T


it just so happens to be that 'static' function variables in D 
require compile-time available initializers. To initialize with a 
runtime value on first execution, you have to implement that 
manually:

void foo(int a)
{
     static int first;
     static first_initialized = false;

     if(!first_initialized)
     {
         first = a;
         first_initialized = true;
     }

     // ...
}
Apr 21 2012
prev sibling parent reply "SomeDude" <lovelydear mailmetrash.com> writes:
On Saturday, 21 April 2012 at 14:40:00 UTC, H. S. Teoh wrote:

 On Sat, Apr 21, 2012 at 03:12:05PM +0400, Dmitry Olshansky 
 wrote:

 On 21.04.2012 14:48, SomeDude wrote:

On Saturday, 21 April 2012 at 10:21:49 UTC, Dmitry Olshansky 
wrote:

Just stop using ctRegex for now... it's experimental.

Or more to the point the problem is this. I've seen this one 
on bugzilla:

version(CtRgx) {
enum Re = ctRegex!re;//auto is OK here BTW
} else {//that's the problem. It's _parsed_ at compile-time
static Re = regex(re);//switch static to auto
}
}

And there is little I can do untill CTFE stops bleeding RAM.


Well, neither of those works:

version(CtRgx) {
auto Re = ctRegex!re;//auto is OK here BTW
} else {//that's the problem. It's _parsed_ at compile-time
auto Re = regex(re);//switch static to auto
}




 [...]

 Hmph. I should've checked dmd memory usage when I wrote that. 
 :-(

 But anyway, even on my souped up AMD hexacore system, the 
 ctRegex
 version takes significantly longer to compile than the 
 non-ctRegex
 version. Perhaps I should just avoid ctRegex for now (though it 
 *is* an
 ultracool feature of std.regex).

 T


Well, the big problem is, even if I fall back to runtime regex, I 
can't compile anymore on a Windows box with 2Gb of RAM. It's hard 
to swallow...
Apr 22 2012
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Apr 22, 2012 at 12:31:16PM +0200, SomeDude wrote:

 On Saturday, 21 April 2012 at 14:40:00 UTC, H. S. Teoh wrote:


[...]

Hmph. I should've checked dmd memory usage when I wrote that. :-(

But anyway, even on my souped up AMD hexacore system, the ctRegex
version takes significantly longer to compile than the non-ctRegex
version. Perhaps I should just avoid ctRegex for now (though it *is*
an ultracool feature of std.regex).


 
 Well, the big problem is, even if I fall back to runtime regex, I
 can't compile anymore on a Windows box with 2Gb of RAM. It's hard to
 swallow...


It's my fault. I really should be using module globals for those
regexes, and a module ctor (static this) for initializing them.

It would be nice if the CTFE implementation was improved, though.  CTFE
is one of the big major features of D that I really liked.

But speaking of which, are you using the latest version of dmd? 'cos I
think recently there were some CTFE efficiency issues that got fixed.


T

-- 
"How are you doing?" "Doing what?"
Apr 22 2012
parent "Nick Sabalausky" <SeeWebsiteToContactMe semitwist.com> writes:
"H. S. Teoh" <hsteoh quickfur.ath.cx> wrote in message 
news:mailman.2061.1335131543.4860.digitalmars-d puremagic.com...

 It's my fault. I really should be using module globals for those
 regexes, and a module ctor (static this) for initializing them.


In most cases, I've come to prefer lazy initalization (via a module-level 
 property) over module ctors because:

1. If it never actually gets used, you avoid adding extra processing at 
startup merely because you imported some module.

2. It decreases the risk of hitting the dreaded cyclic-module-dependency 
error (major PITA).


you can still do that by simply accessing it at the beginning of main.

I've always felt module ctors were a great idea, but after hitting the 
cyclic dependency issue enough times (or even just the first time), I've 
come to think they're, unfortunately, best avoided.
Apr 22 2012
prev sibling parent reply "nhk" <nhkcon googlemail.com> writes:
Please bear with my ignorance I'm new to D, but why is that any
better compared to a simple

switch(key){
default: throw Exception("Invalid attribute '%s'".format(key));
case "name": d.name = value;
                break;
...
...
}



On Friday, 20 April 2012 at 04:05:43 UTC, H. S. Teoh wrote:

 I'm writing some code that does some very simplistic parsing, 
 and I'm
 just totally geeking out on how awesome D is for writing such 
 code:

 	import std.conv;
 	import std.regex;
 	import std.stdio;

 	struct Data {
 		string name;
 		string phone;
 		int age;
 		... // a whole bunch of other stuff
 	}

 	void main() {
 		Data d;
 		foreach (line; stdin.byLine()) {
 			auto m = match(line, "(\w+)\s+(\w+)");
 			if (!m) continue;

 			auto key = m.captures[1];
 			auto value = m.captures[2];

 			alias void delegate(string key, string value) attrDg;
 			attrDg[string] dgs = [
 				"name": delegate(string key, string value) {
 					d.name = value;
 				},
 				"phone": delegate(string key, string value) {
 					d.phone = value;
 				},
 				"age": delegate(string key, string value) {
 					d.age = to!int(value);
 				},
 				...	// whole bunch of other stuff to
 					// parse different attributes
 			];
 			attrDg errordg = delegate(string key, string value) {
 				throw Exception("Invalid attribute '%s'"
 					.format(key));
 			};

 			// This is pure awesomeness:
 			dgs.get(key.idup, errordg)(key.idup, value.idup);
 		}
 		// ... do something with Data
 	}

 Basically, I use std.regex to extract keywords from the input, 
 then use
 an AA to map keywords to code that implement said keyword.  
 That AA of
 delegates is just pure awesomeness. AA.get's default value 
 parameter
 lets you process keywords and handle errors with a single AA 
 lookup.  I
 mean, this is even better than Perl for this kind of 
 text-processing
 code!

 The only complaint is that I couldn't write auto[string] dgs 
 and have
 the compiler auto-infer the delegate type. :-) Additionally, I 
 wasn't
 sure if I could omit the "delegate(string,string)" after each 
 keyword;
 if that's actually allowed, then this would make D totally pwn 
 Perl!!

 (I left out some stuff that makes this code even more of a joy 
 to write:
 using nested try/catch blocks, I can throw exceptions from 
 deep-down
 parsing code and have the loop that loops over input lines 
 automatically
 prefix error messages with the filename/line number where the 
 error
 occurred. This way, even errors thrown by to!int() will be 
 formatted
 nicely. With Perl, this gets extremely messy due to its 
 pathological use
 of $. for line numbers which can get overwritten in unexpected 
 places if
 you're processing more than one file at a time.)

 Did I mention I'm totally in love with D?? Seriously. It can 
 handle
 system-level code and "high-level" text-processing code with 
 total
 impunity. What's there not to like?!


 T
Apr 21 2012
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sat, Apr 21, 2012 at 06:17:58PM +0200, nhk wrote:

 Please bear with my ignorance I'm new to D, but why is that any
 better compared to a simple
 
 switch(key){
 default: throw Exception("Invalid attribute '%s'".format(key));
 case "name": d.name = value;
                break;
 ...
 ...
 }


Firstly, the current version of dmd doesn't do any special translation
of a switch statement on a string variable, so it just becomes a
sequence of if-statements with string comparisons (slow). Using an
associative array makes the lookup O(1), which is fast(er) when you have
a lot of keys.

Second, using an AA allowed me to factor out the code that does the
parsing (see the parseBlock function in the second version of my code).
There's no way to do this with a switch statement.

In retrospect, the second point is probably more important, because the
use of delegates probably delays the point at which AA performance
starts to overtake a switch statement.


T

-- 
Caffeine underflow. Brain dumped.
Apr 21 2012