www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - ubyte[] -> immutable(ubyte)[]

reply Andrej Mitrovic <andrej.mitrovich test.com> writes:
This is from TDPL page 407:

import std.algorithm,
       std.concurrency,
       std.stdio;

void main()
{
    enum bufferSize = 1024 * 100;
    auto tid = spawn(&fileWriter);
    
    // Read loop
    foreach (immutable(ubyte)[] buffer; stdin.byChunk(bufferSize))
    {
        send(tid, buffer);
    }
}

void fileWriter()
{
    // write loop
    while (true)
    {
        auto buffer = receiveOnly!(immutable(ubyte)[])();
        tgt.write(buffer);
    }
}

Error:
C:\DMD\dmd2\windows\bin\..\..\src\phobos\std\stdio.d(1943):
Error: cannot implicitly convert expression (buffer) of type ubyte[] 
to immutable(ubyte)[]

Yet interestingly I can't use type inference:

    foreach (buffer; stdin.byChunk(bufferSize))
    {
        send(tid, buffer);
    }

Error: stdin_stdout_copy.d(11): Error: cannot infer type for buffer

But in the original code I get back a mutable ubyte[] which I can't implicitly
convert to immutable (I'd need a copy first). There's no .dup or .idup property
for stdin.byChunk. So what am I supossed to do here?
Sep 03 2010
parent reply Andrej Mitrovic <test test.test> writes:
 I'm trying to use algorithm.copy, but I get back nothing in the copy buffer.
How do I to copy an array of ubyte's?

iimport std.algorithm,
       std.concurrency,
       std.stdio;

void main()
{
    enum bufferSize = 4;
    auto tid = spawn(&fileWriter);
    
    // Read loop
    foreach (ubyte[] buffer; stdin.byChunk(bufferSize))
    {
        immutable(ubyte)[] copy_buffer;
        copy(buffer, copy_buffer);
        
        writeln(copy_buffer);  // writes nothing

        send(tid, copy_buffer);
    }
}

void fileWriter()
{
    while (true)
    {
        auto buffer = receiveOnly!(immutable(ubyte)[])();
        // writeln(buffer);
    }
}

Andrej Mitrovic Wrote:

 This is from TDPL page 407:
 
 import std.algorithm,
        std.concurrency,
        std.stdio;
 
 void main()
 {
     enum bufferSize = 1024 * 100;
     auto tid = spawn(&fileWriter);
     
     // Read loop
     foreach (immutable(ubyte)[] buffer; stdin.byChunk(bufferSize))
     {
         send(tid, buffer);
     }
 }
 
 void fileWriter()
 {
     // write loop
     while (true)
     {
         auto buffer = receiveOnly!(immutable(ubyte)[])();
         tgt.write(buffer);
     }
 }
 
 Error:
 C:\DMD\dmd2\windows\bin\..\..\src\phobos\std\stdio.d(1943):
 Error: cannot implicitly convert expression (buffer) of type ubyte[] 
 to immutable(ubyte)[]
 
 Yet interestingly I can't use type inference:
 
     foreach (buffer; stdin.byChunk(bufferSize))
     {
         send(tid, buffer);
     }
 
 Error: stdin_stdout_copy.d(11): Error: cannot infer type for buffer
 
 But in the original code I get back a mutable ubyte[] which I can't implicitly
convert to immutable (I'd need a copy first). There's no .dup or .idup property
for stdin.byChunk. So what am I supossed to do here?
 
 
Sep 09 2010
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Andrej Mitrovic:
  I'm trying to use algorithm.copy, but I get back nothing in the copy buffer.
How do I to copy an array of ubyte's?
a[] = b[]; Bye, bearophile
Sep 09 2010
parent Andrej Mitrovic <test test.test> writes:
bearophile Wrote:

 Andrej Mitrovic:
  I'm trying to use algorithm.copy, but I get back nothing in the copy buffer.
How do I to copy an array of ubyte's?
a[] = b[]; Bye, bearophile
No, that wouldn't work. It complains about conversion from mutable to immutable. idup works fine though.
Sep 10 2010
prev sibling next sibling parent reply Kagamin <spam here.lot> writes:
Andrej Mitrovic Wrote:

     foreach (ubyte[] buffer; stdin.byChunk(bufferSize))
     {
         immutable(ubyte)[] copy_buffer;
         copy(buffer, copy_buffer);
         
         writeln(copy_buffer);  // writes nothing
 
         send(tid, copy_buffer);
     }
Isn't destination the left argument? Why you don't use send(tid, buffer.idup); ?
Sep 09 2010
parent reply Andrej Mitrovic <test test.test> writes:
Yeah, one would think the destination is on the left (just like the standard C
way of doing it), but it's not. I checked it in the docs and the source. And
idup works, thanks.

Kagamin Wrote:

 Andrej Mitrovic Wrote:
 
     foreach (ubyte[] buffer; stdin.byChunk(bufferSize))
     {
         immutable(ubyte)[] copy_buffer;
         copy(buffer, copy_buffer);
         
         writeln(copy_buffer);  // writes nothing
 
         send(tid, copy_buffer);
     }
Isn't destination the left argument? Why you don't use send(tid, buffer.idup); ?
Sep 10 2010
parent reply vino <vino_ss hotmail.com> writes:
On Friday, 10 September 2010 at 15:15:38 UTC, Andrej Mitrovic 
wrote:
 Yeah, one would think the destination is on the left (just like 
 the standard C way of doing it), but it's not. I checked it in 
 the docs and the source. And idup works, thanks.

 Kagamin Wrote:

 Andrej Mitrovic Wrote:
 
     foreach (ubyte[] buffer; stdin.byChunk(bufferSize))
     {
         immutable(ubyte)[] copy_buffer;
         copy(buffer, copy_buffer);
 
         writeln(copy_buffer);  // writes nothing
 
         send(tid, copy_buffer);
     }
Isn't destination the left argument? Why you don't use send(tid, buffer.idup); ?
Hi Andrei/All, Request your help, I am trying the program written by you with few changes such as the example program read the input from stdin and prints the data to stdout, but my program reads the input from the file(readfile.txt) and writes the output to another file(writefile.txt), and I am getting the below errors while compiling Testing: Read a file(readfile.txt : Contain 20,000 lines) by chunk into a buffer(read buffer) Pass the buffered data to an output buffer(write buffer) and then write the output buffer to a to another file(writefile.txt). Error: [root localhost DProjects]# dmd readwriteb.d readwriteb.d(7): Error: cannot implicitly convert expression (__aggr2859.front()) of type ubyte[] to immutable(ubyte)[] readwriteb.d(15): Error: cannot implicitly convert expression (receiveOnly()) of type immutable(ubyte)[] to std.outbuffer.OutBuffer [root localhost DProjects]# Version: DMD64 D Compiler v2.071.0 Code: import std.algorithm, std.concurrency, std.stdio, std.outbuffer, std.file; void main() { enum bufferSize = 1024 * 100; auto file = File("readfile.txt", "r"); auto tid = spawn(&fileWriter); foreach (immutable(ubyte)[] buffer; file.byChunk(bufferSize)) { send(tid, buffer); } } void fileWriter() { auto wbuf = new OutBuffer(); for (;;) { wbuf = receiveOnly!(immutable(ubyte)[])(); write("writefile.txt", wbuf); } } After few changes as below I was able to compile, but when i run the program there is no data in the writefile.txt import std.algorithm, std.concurrency, std.stdio, std.file; void main() { enum bufferSize = 1024 * 100; auto file = File("readfile.txt", "r"); auto tid = spawn(&fileWriter); foreach (ubyte[] buffer; file.byChunk(bufferSize)) { send(tid, buffer.idup); } } void fileWriter() { auto file = File("writefile.txt", "w"); for (;;) { auto wbuf = receiveOnly!(ubyte[])(); file.writeln("writefile.txt", wbuf); } } From, Vino.B
May 03 2016
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 5/3/16 5:31 PM, vino wrote:
 On Friday, 10 September 2010 at 15:15:38 UTC, Andrej Mitrovic wrote:
 Yeah, one would think the destination is on the left (just like the
 standard C way of doing it), but it's not. I checked it in the docs
 and the source. And idup works, thanks.

 Kagamin Wrote:

 Andrej Mitrovic Wrote:

     foreach (ubyte[] buffer; stdin.byChunk(bufferSize))
     {
         immutable(ubyte)[] copy_buffer;
         copy(buffer, copy_buffer);
         writeln(copy_buffer);  // writes nothing
         send(tid, copy_buffer);
}
Isn't destination the left argument? Why you don't use send(tid, buffer.idup); ?
Hi Andrei/All, Request your help, I am trying the program written by you with few changes such as the example program read the input from stdin and prints the data to stdout, but my program reads the input from the file(readfile.txt) and writes the output to another file(writefile.txt), and I am getting the below errors while compiling Testing: Read a file(readfile.txt : Contain 20,000 lines) by chunk into a buffer(read buffer) Pass the buffered data to an output buffer(write buffer) and then write the output buffer to a to another file(writefile.txt). Error: [root localhost DProjects]# dmd readwriteb.d readwriteb.d(7): Error: cannot implicitly convert expression (__aggr2859.front()) of type ubyte[] to immutable(ubyte)[] readwriteb.d(15): Error: cannot implicitly convert expression (receiveOnly()) of type immutable(ubyte)[] to std.outbuffer.OutBuffer [root localhost DProjects]# Version: DMD64 D Compiler v2.071.0 Code: import std.algorithm, std.concurrency, std.stdio, std.outbuffer, std.file; void main() { enum bufferSize = 1024 * 100; auto file = File("readfile.txt", "r"); auto tid = spawn(&fileWriter); foreach (immutable(ubyte)[] buffer; file.byChunk(bufferSize)) { send(tid, buffer); } } void fileWriter() { auto wbuf = new OutBuffer(); for (;;) { wbuf = receiveOnly!(immutable(ubyte)[])(); write("writefile.txt", wbuf); } } After few changes as below I was able to compile, but when i run the program there is no data in the writefile.txt import std.algorithm, std.concurrency, std.stdio, std.file; void main() { enum bufferSize = 1024 * 100; auto file = File("readfile.txt", "r"); auto tid = spawn(&fileWriter); foreach (ubyte[] buffer; file.byChunk(bufferSize)) { send(tid, buffer.idup); } } void fileWriter() { auto file = File("writefile.txt", "w"); for (;;) { auto wbuf = receiveOnly!(ubyte[])();
buffer.idup turns into immutable(ubyte)[]. You must receive this type: auto wbuf = receiveOnly!(immutable(ubyte)[])(); What is likely happening is that this receiveOnly throws an exception you aren't catching, and then the program hangs. Try catching any errors/exceptions in your fileWriter thread function and print them out to see what is really happening. -Steve
May 04 2016
prev sibling parent reply Pelle <pelle.mansson gmail.com> writes:
On 09/10/2010 04:40 AM, Andrej Mitrovic wrote:
   I'm trying to use algorithm.copy, but I get back nothing in the copy buffer.
How do I to copy an array of ubyte's?

 iimport std.algorithm,
         std.concurrency,
         std.stdio;

 void main()
 {
      enum bufferSize = 4;
      auto tid = spawn(&fileWriter);

      // Read loop
      foreach (ubyte[] buffer; stdin.byChunk(bufferSize))
      {
          immutable(ubyte)[] copy_buffer;
          copy(buffer, copy_buffer);

          writeln(copy_buffer);  // writes nothing

          send(tid, copy_buffer);
      }
 }

 void fileWriter()
 {
      while (true)
      {
          auto buffer = receiveOnly!(immutable(ubyte)[])();
          // writeln(buffer);
      }
 }

 Andrej Mitrovic Wrote:

 This is from TDPL page 407:

 import std.algorithm,
         std.concurrency,
         std.stdio;

 void main()
 {
      enum bufferSize = 1024 * 100;
      auto tid = spawn(&fileWriter);

      // Read loop
      foreach (immutable(ubyte)[] buffer; stdin.byChunk(bufferSize))
      {
          send(tid, buffer);
      }
 }

 void fileWriter()
 {
      // write loop
      while (true)
      {
          auto buffer = receiveOnly!(immutable(ubyte)[])();
          tgt.write(buffer);
      }
 }

 Error:
 C:\DMD\dmd2\windows\bin\..\..\src\phobos\std\stdio.d(1943):
 Error: cannot implicitly convert expression (buffer) of type ubyte[]
 to immutable(ubyte)[]

 Yet interestingly I can't use type inference:

      foreach (buffer; stdin.byChunk(bufferSize))
      {
          send(tid, buffer);
      }

 Error: stdin_stdout_copy.d(11): Error: cannot infer type for buffer

 But in the original code I get back a mutable ubyte[] which I can't implicitly
convert to immutable (I'd need a copy first). There's no .dup or .idup property
for stdin.byChunk. So what am I supossed to do here?
std.algorithm.copy will copy an input range into an output range. An array is a valid output range, but does not append as you seem to expect. Instead, it fills the array. int[] a = new int[](3); copy([1,2,3],a); assert (a == [1,2,3]); To get an output range which appends to an array, use appender. In this case, however, you simply want buffer.idup; :-)
Sep 10 2010
parent Andrej Mitrovic <test test.test> writes:
Ah, idup. Too obvious, but I missed it. Thanks.

Pelle Wrote:

 std.algorithm.copy will copy an input range into an output range. An 
 array is a valid output range, but does not append as you seem to 
 expect. Instead, it fills the array.
 
 int[] a = new int[](3);
 copy([1,2,3],a);
 assert (a == [1,2,3]);
 
 To get an output range which appends to an array, use appender.
 
 In this case, however, you simply want buffer.idup; :-)
Sep 10 2010