www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Thread-safety and lazy-initialization of libraries

reply "Sergey Protko" <fesors gmail.com> writes:
For some research i decided to write small high-level binding for 
libmpg123.

The question is how to write thread-safe lazy-initialization of 
library.

libmpg123 has mpg123_init and mpg123_exit functions, which are 
not thread-safe, so we should to call them only once per process. 
Most of useful libraries also has such stuff. But manual 
initialization is killing all beauty of high-level bindings.

Is there any proper way to do on-demand lazy-initialization of 
used library, which will be also thread-safe? How do i need to 
handle cases where some methods, which requires library to be 
initialized, called from different threads at the same time?

Thanks for your answers.
Jun 30 2014
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Sergey Protko:


 libmpg123 has mpg123_init and mpg123_exit functions, which are 
 not thread-safe, so we should to call them only once per 
 process. Most of useful libraries also has such stuff. But 
 manual initialization is killing all beauty of high-level 
 bindings.


I think module "static this" is thread-local, so in theory you 
can use that. But I don't know if it's a good idea to perform 
heavy computations inside those module static this.

Bye,
bearophile
Jun 30 2014
next sibling parent reply "Sergey Protko" <fesors gmail.com> writes:
On Monday, 30 June 2014 at 21:05:32 UTC, bearophile wrote:

 Sergey Protko:


 libmpg123 has mpg123_init and mpg123_exit functions, which are 
 not thread-safe, so we should to call them only once per 
 process. Most of useful libraries also has such stuff. But 
 manual initialization is killing all beauty of high-level 
 bindings.


 I think module "static this" is thread-local, so in theory you 
 can use that. But I don't know if it's a good idea to perform 
 heavy computations inside those module static this.

 Bye,
 bearophile


I thought about this. But static constructors doesn't solve 
problem with on-demand initialization in case, where there is 
several classes. For example Decoder and Encoder both requires 
library to be initialized before they are actually be used.
Jun 30 2014
parent reply "Rene Zwanenburg" <renezwanenburg gmail.com> writes:
On Monday, 30 June 2014 at 21:32:34 UTC, Sergey Protko wrote:

 On Monday, 30 June 2014 at 21:05:32 UTC, bearophile wrote:

 Sergey Protko:


 libmpg123 has mpg123_init and mpg123_exit functions, which 
 are not thread-safe, so we should to call them only once per 
 process. Most of useful libraries also has such stuff. But 
 manual initialization is killing all beauty of high-level 
 bindings.


 I think module "static this" is thread-local, so in theory you 
 can use that. But I don't know if it's a good idea to perform 
 heavy computations inside those module static this.

 Bye,
 bearophile


 I thought about this. But static constructors doesn't solve 
 problem with on-demand initialization in case, where there is 
 several classes. For example Decoder and Encoder both requires 
 library to be initialized before they are actually be used.


Use a shared module constructor? It's called only once, not 
per-thread.


module mpg123;

shared static this()
{
   mpg123_init();
}

shared static ~this()
{
   mpg123_exit();
}
Jun 30 2014
parent "Sergey Protko" <fesors gmail.com> writes:
On Monday, 30 June 2014 at 21:36:10 UTC, Rene Zwanenburg wrote:

 On Monday, 30 June 2014 at 21:32:34 UTC, Sergey Protko wrote:

 On Monday, 30 June 2014 at 21:05:32 UTC, bearophile wrote:

 Sergey Protko:


 libmpg123 has mpg123_init and mpg123_exit functions, which 
 are not thread-safe, so we should to call them only once per 
 process. Most of useful libraries also has such stuff. But 
 manual initialization is killing all beauty of high-level 
 bindings.


 I think module "static this" is thread-local, so in theory 
 you can use that. But I don't know if it's a good idea to 
 perform heavy computations inside those module static this.

 Bye,
 bearophile


 I thought about this. But static constructors doesn't solve 
 problem with on-demand initialization in case, where there is 
 several classes. For example Decoder and Encoder both requires 
 library to be initialized before they are actually be used.


 Use a shared module constructor? It's called only once, not 
 per-thread.


 module mpg123;

 shared static this()
 {
   mpg123_init();
 }

 shared static ~this()
 {
   mpg123_exit();
 }


Oh, sorry. I doesn't thought about module constructors. But how 
we should handle errors on initialization? I can't just throw an 
exception from module constructor. Also this way isn't lazy at 
all)

I could write something like thread-safe singleton with lazy 
initialization, which returns status of library (OK or error 
code), but i not sure about that. Is it reasonable way?
Jun 30 2014
prev sibling parent reply Mike Wey <mike-wey example.com> writes:
On 06/30/2014 11:05 PM, bearophile wrote:

 Sergey Protko:


 libmpg123 has mpg123_init and mpg123_exit functions, which are not
 thread-safe, so we should to call them only once per process. Most of
 useful libraries also has such stuff. But manual initialization is
 killing all beauty of high-level bindings.


 I think module "static this" is thread-local, so in theory you can use
 that. But I don't know if it's a good idea to perform heavy computations
 inside those module static this.

 Bye,
 bearophile


You'll need to use a `shared static this` if those functions can be 
called only once per process.
A regular static this is executed once per Thread.

-- 
Mike Wey
Jun 30 2014
parent reply "H. S. Teoh via Digitalmars-d-learn" <digitalmars-d-learn puremagic.com> writes:
On Mon, Jun 30, 2014 at 11:36:21PM +0200, Mike Wey via Digitalmars-d-learn
wrote:

 On 06/30/2014 11:05 PM, bearophile wrote:

Sergey Protko:


libmpg123 has mpg123_init and mpg123_exit functions, which are not
thread-safe, so we should to call them only once per process. Most
of useful libraries also has such stuff. But manual initialization
is killing all beauty of high-level bindings.


I think module "static this" is thread-local, so in theory you can
use that. But I don't know if it's a good idea to perform heavy
computations inside those module static this.

Bye,
bearophile


 
 You'll need to use a `shared static this` if those functions can be
 called only once per process.
 A regular static this is executed once per Thread.


[...]

Depending on how lazy you want initialization to be, you might want to
consider using __gshared (for process-global state) with appropriate
synchronization locks to make sure threads don't stomp over each other.
Then you can check if component X has been initialized (per process)
each time a thread calls some function that depends on X, and if it is,
initialize it, if not, just do nothing (or return the global instance).


T

-- 
Verbing weirds language. -- Calvin (& Hobbes)
Jun 30 2014
parent "Sergey Protko" <fesors gmail.com> writes:
On Monday, 30 June 2014 at 21:55:56 UTC, H. S. Teoh via 
Digitalmars-d-learn wrote:

 On Mon, Jun 30, 2014 at 11:36:21PM +0200, Mike Wey via 
 Digitalmars-d-learn wrote:

 On 06/30/2014 11:05 PM, bearophile wrote:

Sergey Protko:


libmpg123 has mpg123_init and mpg123_exit functions, which 
are not
thread-safe, so we should to call them only once per 
process. Most
of useful libraries also has such stuff. But manual 
initialization
is killing all beauty of high-level bindings.


I think module "static this" is thread-local, so in theory 
you can
use that. But I don't know if it's a good idea to perform 
heavy
computations inside those module static this.

Bye,
bearophile


 
 You'll need to use a `shared static this` if those functions 
 can be
 called only once per process.
 A regular static this is executed once per Thread.


 [...]

 Depending on how lazy you want initialization to be, you might 
 want to
 consider using __gshared (for process-global state) with 
 appropriate
 synchronization locks to make sure threads don't stomp over 
 each other.
 Then you can check if component X has been initialized (per 
 process)
 each time a thread calls some function that depends on X, and 
 if it is,
 initialize it, if not, just do nothing (or return the global 
 instance).


 T


Something like this?

module libmpg123;

__gshared bool initialized_;
static bool initializedTLS;

void init_requires()
{
     if (!initializedTLS) {
         synchronized {
             if (!initialized_) {
                 // todo: handle errors?
                 mpg123_init();
                 initialized_ = true;
             }
             initializedTLS = initialized_;
         }
     }
}

Well, i'll try... Thank you.
Jun 30 2014
prev sibling next sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 06/30/2014 01:53 PM, Sergey Protko wrote:


 The question is how to write thread-safe lazy-initialization of library.


David Simcha's DConf 2013 presentation covers this question. At around 
minute 28:

 
https://www.youtube.com/watch?feature=player_detailpage&v=yMNMV9JlkcQ#t=1690

Unfortunately, he never published his slides.

Ali
Jun 30 2014
parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 06/30/2014 03:46 PM, Ali Çehreli wrote:


 https://www.youtube.com/watch?feature=player_detailpage&v=yMNMV9JlkcQ#t=1690


 Unfortunately, he never published his slides.


Here is the information about an article he wrote on the topic:

 
http://forum.dlang.org/thread/pelhvaxwjzhehdjtpsav forum.dlang.org#post-pelhvaxwjzhehdjtpsav:40forum.dlang.org

Ali
Jun 30 2014
prev sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
On Monday, 30 June 2014 at 20:53:25 UTC, Sergey Protko wrote:

 Is there any proper way to do on-demand lazy-initialization of 
 used library, which will be also thread-safe? How do i need to 
 handle cases where some methods, which requires library to be 
 initialized, called from different threads at the same time?


pthread_once comes to mind.
Jul 01 2014