www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - maybe i got a bug

reply "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
I have into FastqReader structure a member named _number and for 
a strange reason his property return always 0 while into popFront 
where i do some some operation with, the number is correctly 
increased: 0,1,2,3 not 0,1,0,1


----------------------------
$ grep _number fastq.d
fastq.d:82:        size_t  _number;
fastq.d:108:                _number++;
fastq.d:113:                    _number++;
fastq.d:162:            return _number;


----------------------------
gdb --args ./fastq ~/Projets/little.fastq
(gdb) b 108
Breakpoint 1 at 0x403ad2: file fastq.d, line 108.
(gdb) b 114
Breakpoint 2 at 0x403b3c: file fastq.d, line 114.
(gdb) b 162
Breakpoint 3 at 0x404022: file fastq.d, line 162.
(gdb) r
Breakpoint 1, fastq.FastqReader.front (this=...) at fastq.d:108
108	                _number++;
(gdb) info args
this = {_currentState = void, _position = 0, _letters = 0, 
_letterNumber = 0, _number = 0, _mmFile =  0x2aaaaaad2f00}
(gdb) n
134	            Tuple!( State, dchar ) result;
(gdb) info args
this = {_currentState = void, _position = 0, _letters = 0, 
_letterNumber = 0, _number = 1, _mmFile =  0x2aaaaaad2f00}
(gdb) c
Continuing.

Breakpoint 3, fastq.FastqReader.number (this=...) at fastq.d:162
162	            return _number;
(gdb) info args
this = {_currentState = 0, _position = 0, _letters = 0, 
_letterNumber = 0, _number = 0, _mmFile =  0x2aaaaaad2f00}
(gdb) watch this._number
Hardware watchpoint 5: this._number
(gdb) c
Continuing.
0 IDENTIFIER  

----------------------------

full code here: http://dpaste.dzfl.pl/4f236648
Jan 31 2013
next sibling parent reply "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
When i print the memory addres from _number in front method and 
into number method is not the same while _number should be a 
member of the given instance;

  ./fastq ~/Projets/little.fastq
-> FastqReader.empty()
-> FastqReader.front()
7FFF355EA0C0, 0, 1
-> FastqReader.number()
7FFF355EA090, 0
0 IDENTIFIER  
-> FastqReader.popFront()
-> FastqReader.empty()
-> FastqReader.empty()
-> FastqReader.empty()
-> FastqReader.front()
-> FastqReader.number()
....
7FFF355EA090, 0
0 QUALITY G
-> FastqReader.popFront()
-> FastqReader.empty()
-> FastqReader.empty()
-> FastqReader.empty()
-> FastqReader.empty()
-> FastqReader.empty()
-> FastqReader.front()
7FFF355EA0C0, 1, 2

----------------

here you see that in front _number is located at 7FFF355EA0C0 and 
value was 0 and go to 1
while number method return always 0 and _number is located at 
7FFF355EA090
And later in front method _number had a value of 1 and go to 2 as 
intended
number property continue to return 0
Jan 31 2013
parent reply "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
This problem appear both with gdc / ldc .
i do not have dmd.
So is maybe a dmdfe bug
Jan 31 2013
parent reply "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
I think when i iterate over a phobos range :
   foreach( state, letter; fastq )
and if fastq instance is called into this loop as done at line 
181 is not the same instance

below i show that fastq variable do not have same adress into 
foreach and the followed at line 181

$ gdb --args ./fastq ~/Projets/little.fastq
Reading symbols from /env/export/nfs1/home/jmercier/fastq...done.
(gdb) b 108
Breakpoint 1 at 0x403b9b: file fastq.d, line 108.
(gdb) b 163
Breakpoint 2 at 0x40417f: file fastq.d, line 163.
(gdb) r
Starting program: /env/export/nfs1/home/jmercier/fastq 
/env/cns/home/jmercier/Projets/little.fastq
warning: no loadable sections found in added symbol-file 
system-supplied DSO at 0x2aaaaaaab000
[Thread debugging using libthread_db enabled]
-> FastqReader.empty()
-> FastqReader.front()

Breakpoint 1, fastq.FastqReader.front (this=...) at fastq.d:108
108	                _number++;
(gdb) info args
this = {_currentState = void, _position = 0, _letters = 0, 
_letterNumber = 0, _number = 0, _mmFile =  0x2aaaaaad2f00}
(gdb) p &this._number
$1 = (ulong *) 0x7fffffffd760
(gdb) p &this
$2 = (class FastqReader *) 0x7fffffffd740
(gdb) c
Continuing.
-> FastqReader.number

Breakpoint 2, fastq.FastqReader.number (this=...) at fastq.d:163
163	        }
(gdb) info args
this = {_currentState = 0, _position = 0, _letters = 0, 
_letterNumber = 0, _number = 0, _mmFile =  0x2aaaaaad2f00}
(gdb) p &this._number
$3 = (ulong *) 0x7fffffffd730
(gdb) p &this
$4 = (struct FastqReader *) 0x7fffffffd710
Jan 31 2013
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 01/31/2013 08:59 AM, bioinfornatics wrote:

 I think when i iterate over a phobos range :
 foreach( state, letter; fastq )
 and if fastq instance is called into this loop as done at line 181 is
 not the same instance


Correct. Here is a simpler program:

import std.stdio;

struct S
{
     int i;

      property int front() const
     {
         writefln("front   : %s", &i);
         return i;
     }

      property bool empty() const
     {
         writefln("empty   : %s", &i);
         return i == 0;
     }

     void popFront()
     {
         writefln("popFront: %s", &i);
         --i;
     }
}

void main()
{
     auto s = S(2);
     writefln("main    : %s", &s.i);

     foreach (i; s) {
     }
}

The object in main is a different object:

main    : 7FFFBD429B70
empty   : 7FFFBD429B74
front   : 7FFFBD429B74
popFront: 7FFFBD429B74
empty   : 7FFFBD429B74
front   : 7FFFBD429B74
popFront: 7FFFBD429B74
empty   : 7FFFBD429B74

Ali
Jan 31 2013
parent reply "bioinfornatics" <bioinfornatics gmail.com> writes:
On Thursday, 31 January 2013 at 21:31:08 UTC, Ali Çehreli wrote:

 On 01/31/2013 08:59 AM, bioinfornatics wrote:

 I think when i iterate over a phobos range :
 foreach( state, letter; fastq )
 and if fastq instance is called into this loop as done at


 line 181 is

 not the same instance


 Correct. Here is a simpler program:

 import std.stdio;

 struct S
 {
     int i;

      property int front() const
     {
         writefln("front   : %s", &i);
         return i;
     }

      property bool empty() const
     {
         writefln("empty   : %s", &i);
         return i == 0;
     }

     void popFront()
     {
         writefln("popFront: %s", &i);
         --i;
     }
 }

 void main()
 {
     auto s = S(2);
     writefln("main    : %s", &s.i);

     foreach (i; s) {
     }
 }

 The object in main is a different object:

 main    : 7FFFBD429B70
 empty   : 7FFFBD429B74
 front   : 7FFFBD429B74
 popFront: 7FFFBD429B74
 empty   : 7FFFBD429B74
 front   : 7FFFBD429B74
 popFront: 7FFFBD429B74
 empty   : 7FFFBD429B74

 Ali


is normal or is a bug ?
Jan 31 2013
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 01/31/2013 01:42 PM, bioinfornatics wrote:

 On Thursday, 31 January 2013 at 21:31:08 UTC, Ali Çehreli wrote:

 On 01/31/2013 08:59 AM, bioinfornatics wrote:





 I think when i iterate over a phobos range :
 foreach( state, letter; fastq )
 and if fastq instance is called into this loop as done at


 line 181 is

 not the same instance


 Correct.





 is normal or is a bug ?


It is not a bug. Here is a discussion on the topic:

   http://forum.dlang.org/thread/jp16ni$fug$1 digitalmars.com

Ali
Jan 31 2013
parent "bioinfornatics" <bioinfornatics gmail.com> writes:
On Thursday, 31 January 2013 at 22:02:15 UTC, Ali Çehreli wrote:

 On 01/31/2013 01:42 PM, bioinfornatics wrote:

 On Thursday, 31 January 2013 at 21:31:08 UTC, Ali Çehreli


 wrote:

 On 01/31/2013 08:59 AM, bioinfornatics wrote:





 I think when i iterate over a phobos range :
 foreach( state, letter; fastq )
 and if fastq instance is called into this loop as done at


 line 181 is

 not the same instance


 Correct.





 is normal or is a bug ?


 It is not a bug. Here is a discussion on the topic:

   http://forum.dlang.org/thread/jp16ni$fug$1 digitalmars.com

 Ali




that is insane ...
so the way to go will be to yield at each loop all members value
Jan 31 2013
prev sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
This is not related to your actual problem but I have noticed that you 
have side-effects in your FastqReader.front. I think you will benefit 
from a design where front simply returns the front element and all of 
the side-effects are inside popFront().

Ali
Jan 31 2013
parent reply "bioinfornatics" <bioinfornatics gmail.com> writes:
On Thursday, 31 January 2013 at 22:20:25 UTC, Ali Çehreli wrote:

 This is not related to your actual problem but I have noticed 
 that you have side-effects in your FastqReader.front. I think 
 you will benefit from a design where front simply returns the 
 front element and all of the side-effects are inside popFront().

 Ali


but as fastq instance used to iterate and fastq instance called 
are not same, any fastq method where depend the position in given 
range won't work. you need to return all possible value that you 
could be used
Jan 31 2013
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 01/31/2013 02:27 PM, bioinfornatics wrote:

 On Thursday, 31 January 2013 at 22:20:25 UTC, Ali Çehreli wrote:

 This is not related to your actual problem but I have noticed that you
 have side-effects in your FastqReader.front. I think you will benefit
 from a design where front simply returns the front element and all of
 the side-effects are inside popFront().

 Ali


 but as fastq instance used to iterate and fastq instance called are not
 same, any fastq method where depend the position in given range won't
 work. you need to return all possible value that you could be used


Apparently I didn't understand the code. :)

My comments should be generally correct: Calling front() multiple times 
should return the same element and it should not change the state of the 
range.

Ali
Jan 31 2013
next sibling parent "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
 Apparently I didn't understand the code. :)

 My comments should be generally correct: Calling front() 
 multiple times should return the same element and it should not 
 change the state of the range.

 Ali


my code works like you said you ca
Feb 01 2013
prev sibling parent reply "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
 Apparently I didn't understand the code. :)

 My comments should be generally correct: Calling front() 
 multiple times should return the same element and it should not 
 change the state of the range.

 Ali


My code works like you said you can call front multiple time he 
will return same thing. I check in front the lettern return to 
set or not the state and the section number. this maye should 
move to popFront.

To explain, i iterate over a fastq file with a memory mapped 
file. Then i iterate letter by letter and i need to return the 
given letter and if this letter is wich line type are. I do not 
use \n or \r\n to identify a line as fastq format allow witespace 
and newline int sequence and quality line. Each time i see a new 
identifier line i increase the counter to said at the end they 
are xxx sections in this file.
As is a memmory mapped file i read ubyte, by example 64 is  . 64 
could be a quality letter or the letter to identify indentifiers 
start.
So i need to count how many sequence letter is in this section to 
count number of quality because they are same number.

As i use a memory mapped file i won't copy my struct for able to 
loop as i do not want map the file twice that is rather a big 
problem for a big file this a perf issue. memory mapped file is 
used to read fastly a file so is a nonsense
Feb 01 2013
parent reply "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
My code works like you said you can call front multiple time he
will return same thing. I check in front the letter return to
set or not the state and the section number. this maye should
move to popFront.

To explain, i iterate over a fastq file with a memory mapped
file. Then i iterate letter by letter. I need to return the
given letter and in wich line type are. I do not
use \n or \r\n to identify a line as fastq format allow witespace
and newline into sequence and quality lines. Each time i see a new
identifier line i increase the counter to said at the end they
are xxx sections in this file.
As is a memmory mapped file i read ubyte, by example 64 is  . 64
could be a quality letter or the letter to identify indentifiers
start.
So i need to count how many sequence letters are in this section 
to
count number of quality because they are same number (whispace 
should not to be count but skiped).

As i use a memory mapped file i won't copy my struct for able to
loop as i do not want map the file twice that is rather a big
problem for a big file this a perf issue. memory mapped file is
used to read fastly a file so is a nonsense
Feb 01 2013
next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 1 February 2013 at 09:09:10 UTC, bioinfornatics wrote:

 As i use a memory mapped file i won't copy my struct for able to
 loop as i do not want map the file twice that is rather a big
 problem for a big file this a perf issue. memory mapped file is
 used to read fastly a file so is a nonsense


In a word, the problem is that your "FastqReader" is both 
container and range. Those are complettly different notions that 
should not be confused.

The fastqreader should be a container, that holds your binary 
payload. You should be able to extract a range from the 
container, and iterate on the container, without modifying the 
container.

Currently, iterating on your fastqreader will mutate it. This is 
very bad. The only cases I know of where this happens are with 
pure input ranges, but in those cases, you virtually never have 
access to the underlying container (which usually doesn't exist 
anyways).

--------
The easy workaround is to start by renaming your "popFront" into 
"removeFront": popFront is an iteration primitive. removeFront is 
a container primitive. By changing this name, you fastqreader 
will cease to adhere to any range interface, protecting you from 
wrong usage.

Once there, you need to define a Range, or just have "opSlice()" 
and "opSlice(size_t, size_t)" return said range.

Ideally, I'd avoid defining an actual "Range" type, and simply 
return the string (as you are doing).

However, given you seem to be doing something with dna, and 
require random access, I'm not sure "string" is the best (string 
is unicode aware, so doesn't actually adhere to RA). I'd stick to 
just returning a "const(ubyte)[]".
Feb 01 2013
prev sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 1 February 2013 at 09:09:10 UTC, bioinfornatics wrote:

 As i use a memory mapped file i won't copy my struct for able to
 loop as i do not want map the file twice that is rather a big
 problem for a big file this a perf issue. memory mapped file is
 used to read fastly a file so is a nonsense


Oh yeah: MmFile is a class, so copying it is free. The problem 
though is that it is shallow, so modying a struct copy will 
(partially) modify the original struct.
Feb 01 2013