www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - DFL background tasks

reply "Scroph" <eight_hf live.fr> writes:
Hello,

I couldn't find a DFL subforum so I decided to post here, I 
apologize in advance if this isn't the right place.

I'm currently using DFL to write a Youtube downloading program 
with a simple GUI. I chose DFL because I enjoy the Entice 
Designer WYSIWYG capabilities, but also because I found DFL to be 
easy to use after reading the sample codes, especially the 
Phonebook example. Unfortunately the project is more or less 
abandoned, and the documentation is incomplete.

To make a long story short, there is a ListView that I want to 
populate with information that could take a while to retrieve 
(video qualities and links), which temporarily freezes the 
program. The way I did things thus far was to simply extend the 
Thread class and pass it the UI elements and let it modify them 
directly once the information is retrieved, or by just running 
the task in a delegate that has access to the elements I wish to 
modify, then passing said delegate to an instance of the Thread 
class. It worked with TextBoxes and Buttons and other elements, 
but not with ListView. I did some reading and it turned out this 
was a frowned upon method, that the UI should only be modified by 
the UI thread, and that this could be achieved with the "Invoke" 

did some more digging and found an Invoke method in the Control 
class, but its signature is hard for me to decipher. I also am 
not sure how to use this with an onClick event.

Control.invoke documentation : 
http://wiki.dprogramming.com/DflDoc/Control-Control-invoke

Any input is appreciated. Thanks in advance.
Jun 07 2015
next sibling parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
For the Invoke call, you should be able to just ignore most the 
arguments and write something like

listview.invoke( delegate Object(Object[]) {
     listview.Add(whatever);
     return null;
});

and it should work.
Jun 07 2015
parent reply "Scroph" <eight_hf live.fr> writes:
On Sunday, 7 June 2015 at 12:30:40 UTC, Adam D. Ruppe wrote:

 For the Invoke call, you should be able to just ignore most the 
 arguments and write something like

 listview.invoke( delegate Object(Object[]) {
     listview.Add(whatever);
     return null;
 });

 and it should work.


Hello Adam, thanks for the fast reply. I tried your code and it 
did work, however it still froze the UI while it was retrieving 
the data.

I'm not sure if I'm using it correctly, I put the invoke call 
inside a Button.click callback. Here's a stripped down version of 
the code : 
https://gist.github.com/Scroph/0f3191fcd8b99db04d4b#file-dfl-d-L14-L17

A few days ago I read abot the XY problem 
(http://xyproblem.info/), so maybe I'm not asking the right 
questions. Here's what I'm actually trying to do : after clicking 
the button, I would like for the UI to remain responsive while :

a) A worker thread retrieves the needed information and places it 
in the ListView or,
b) A worker thread (or an asynchronous task maybe ?) retrieves 
the information and messages the UI with the data when it 
finishes running.

The latter seems to be more idiomatic because it separates the 
"data retrieval" bit from the "UI modification" bit, but I'm not 
sure if DFL does things this way. I tried spawn()ing a worker 
thread from the Button.click callback and receiveOnly(VideoInfo) 
from it, but this too ended up freezing the UI.

Thanks in advance.
Jun 07 2015
parent "thedeemon" <dlang thedeemon.com> writes:
On Sunday, 7 June 2015 at 13:34:59 UTC, Scroph wrote:

 I tried your code and it did work, however it still froze the 
 UI while it was retrieving the data.


Sure, because now you do your long operation in the UI thread. 
What happens in the delegate passed to invoke() happens in the 
main UI thread.

You've got two steps: first some long operation producing some 
data, then updating the listview with this data. You need to do 
the long processing part in a worker thread and updating in the 
UI thread. This is how it's done:

in the button callback (which runs in UI thread) you spawn() a 
thread with some worker function. This worker function does your 
long operation, and it happens in separate thread so the UI keeps 
responding. After you finished this long operation you call 
listview.invoke() with a delegate that just updates the listview, 
it doesn't do any long operations. This update will happen in the 
UI thread.

Something like:

void loadDetailsCallback(Control sender, EventArgs ea)
{
   ...
   spawn(&loadDetails, qualityBox);  // quickly launch a thread
   // and go on responding to UI events
}

void loadDetails(ListView qualityBox) {
   //do the work, this happens in worker thread
   vid = ...

   qualityBox.invoke( { // this part happens in UI thread, quickly
     qualityBox.beginUpdate(); // this belongs out of loop
     foreach(q; vid.qualities) {
       qualityBox.addRow(q.itag.to!string, q.type, q.quality, 
q.link);			
     }
     qualityBox.endUpdate();
   });
}
Jun 08 2015
prev sibling parent reply "thedeemon" <dlang thedeemon.com> writes:
A more general and proper approach is to use message passing from 
std.concurrency. With DFL it looks like this: you spawn() a 
thread and don't pass any GUI controls to it, just thisTid 
(identifier of your main UI thread). In that worker tread when 
you've got some data to show in the UI (be it end result of just 
some status update) you use tid.send(...) and send the data in 
appropriate messages (defined as separate structs or classes). In 
the main UI thread you've got a Timer running that checks whether 
there are any messages in the main thread's mailbox and process 
them there.

Here's an example from a real DFL project:
https://bitbucket.org/infognition/undup/src/e8d295b89bc76545860e38a8c9ee171c86f3c84c/newscan.d?at=default#cl-200

OnStart() is a button callback. It does some quick UI updates and 
spawns a thread, passing relevant data and thisTid:

   worker = spawn(&makeScan, fname, hdr, thisTid);

(makeScan is a function with some long running operation, it's 
defined in another module)

There is also a timer set up when a form is created, and in the 
timer function OnTimer() there is a check for new messages via 
receiveTimeout(dur!"msecs"(0).

while(receiveTimeout(dur!"msecs"(0), &RcvMsgNumOfDirs, 
&RcvMsgScanning, &RcvMsgDone)) {}

Important part here is to make it non-blocking, it should not sit 
here waiting for new messages, otherwise UI will not be 
responsive.

And RcvMsgNumOfDirs, RcvMsgScanning, RcvMsgDone are functions 
that react to corresponding messages sent from the worker thread. 
They work in the UI thread, of course.
Jun 08 2015
next sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
Just a couple of minor improvements:

On 06/08/2015 12:22 AM, thedeemon wrote:


 spawn() a thread and don't pass any GUI controls to it, just
 thisTid (identifier of your main UI thread)


The owner's thread id is available as 'ownerTid' as well.


 receiveTimeout(dur!"msecs"(0).


UFCS makes durations more pleasant: :)

     receiveTimeout(0.msecs)

Ali
Jun 08 2015
parent "thedeemon" <dlang thedeemon.com> writes:
On Monday, 8 June 2015 at 07:45:28 UTC, Ali Çehreli wrote:


 receiveTimeout(dur!"msecs"(0).


 UFCS makes durations more pleasant: :)
     receiveTimeout(0.msecs)


Oh, thanks! I missed this bit. This is much nicer!
Jun 08 2015
prev sibling parent reply "Scroph" <eight_hf live.fr> writes:
Briliant, thanks a lot ! Looks like I misunderstood Adam's reply, 
sorry about that !

I tried different things but I didn't think of calling invoke 
from within the worker thread, that solved the freezing problem. 
I ended up using the Thread class; spawn complained about the 
mutability of the given arguments. I realize what I did may be an 
ugly solution for the problem at hand, so I'll try to modify the 
code in order to avoid giving the thread UI elements altogether, 
thus being able to take advantage of more.. modern techniques 
like std.concurrency.

I'm using the Timer trick on another part of the project. Right 
now I'm running instances of a "Downloader" class in the 
background, a class that extends Thread and provides some useful 
functionality : a way to track the progress, to set rate limits, 
to abort the download, etc. It seems to be working fine, I'm 
using the (incomplete) program to download Visual Studio as we 
speak.

I set up a timer to periodically retrieve the progress of a 
download from the instances then update the ListView accordingly. 
Right now I'm hesitating between continuing with this approach vs 
using std.concurrency or even std.parallelism since the downloads 
are supposed to run in parallel without communicating with one 
another, but the only way I see would be to place the message 
handling code inside a loop, which for some reason doesn't sit 
right with me. I'm still learning concurrency (courtesy of Ali 
Çehreli and his book), so maybe my doubts are unfounded. Here's 
what I'll probably end up using, minus the error handling :

void dlWorker(HTTP client, string local, string remote, bool 
resume = true)
{
	auto fh = File(local, resume ? "a" : "w");
	scope(exit)
		fh.close;
	size_t size;
	size_t progress;
	client.onProgress = (size_t dlTotal, size_t dlNow, size_t 
ulTotal, size_t ulNow) {
		size = dlTotal;
		progress = dlNow;
	};
	client.onReceive = (ubyte[] data) {
		fh.rawWrite(data);
		return data.length;
	};	
	client.perform;
	while(!client.isStopped)
	{
		auto msg = receiveOnly!(MsgType, int)();
		switch(msg[0])
		{
			case MsgType.progress: send(ownerTid, progress); break;
			case MsgType.size: send(ownerTid, size); break;
			case MsgType.abort:
				client.shutdown;
				send(ownerTid, 1);
			break;
			default:
				client.handle.set(CurlOption.recv_rate_speed_large, msg[1]);
				send(ownerTid, 1);
			break;
		}
	}
}

enum MsgType
{
	progress, size, abort, limit;
}

Though I'm not entirely sure how I would tie the ListView rows to 
their respective workers. But I'll cross that bridge when I get 
to it.

Again, thanks for all the help. You guys are definitely going to 
be mentioned in the "About" section of the finished product.
Jun 10 2015
parent "thedeemon" <dlang thedeemon.com> writes:
On Wednesday, 10 June 2015 at 22:18:21 UTC, Scroph wrote:

 	client.perform;
 	while(!client.isStopped)


I don't think this will work as you expect. "perform" is a 
synchronous call, it will not return until the download finishes, 
as I understand, so your while loop is too late. I think you 
should insert message processing stuff inside onProgress, though 
it's also suboptimal (if no progress for long time - no joy). 
Proper design will require more thought...
Jun 10 2015