• Andrei Alexandrescu (7/7) Mar 16 2014 A classic idiom for reading lines and keeping them is f.byLine.map!(x =>...
• Jakob Ovrum (8/16) Mar 16 2014 Maybe `byUniqueLine`.
• Andrei Alexandrescu (3/16) Mar 16 2014 No, looks like uniq.
• sclytrack (2/23) Mar 16 2014 f.byLineIdup();
• Jakob Ovrum (4/5) Mar 16 2014 If we can't find a descriptive name, what exactly are we buying
• Andrei Alexandrescu (3/7) Mar 16 2014 A function to send people to, instead of explaining an idiom.
• Andrei Alexandrescu (4/5) Mar 16 2014 Trivial + frequent = good
• Jakob Ovrum (5/10) Mar 16 2014 Fine. I'm guilty of still being in the camp that doesn't think a
• bearophile (17/25) Mar 16 2014 This is essentially this issue, I will reopen it if you want:
• Andrei Alexandrescu (3/5) Mar 16 2014 That will not happen.
• Ary Borenszweig (6/11) Mar 16 2014 Why not?
• Andrei Alexandrescu (3/13) Mar 16 2014 Performance regression.
• Jakob Ovrum (6/21) Mar 16 2014 Not just that, but it would be a breaking change if we make the
• Andrei Alexandrescu (4/6) Mar 16 2014 It introduces the notion of "dup" to newbies. I'd rather go with a
• w0rp (4/4) Mar 16 2014 byLineWithCopy?
• bearophile (8/11) Mar 16 2014 In Python you write:
• Jakob Ovrum (2/9) Mar 16 2014 Is `line` a byte, a character, or a line?
• bearophile (6/7) Mar 16 2014 It's a line, as in all Python programs :-)
• HeiHon (9/13) Mar 16 2014 If the newbie already knows that there is a difference between
• Andrei Alexandrescu (5/20) Mar 16 2014 Depends on the situation. Consider a D tutorial. It would feature little...
• bearophile (4/7) Mar 16 2014 byLineCopy sounds good.
• Marco Leise (5/15) Mar 16 2014 +1. Same reasoning as HeiHon gave.
• Nick Treleaven (4/17) May 21 2014 Implemented and reviewed by monarchdodra. Ready to merge (but more
• bearophile (5/8) May 21 2014 Good. Reopened:
• HeiHon (4/11) Mar 17 2014 Another idea:
• Meta (8/34) Mar 16 2014 Can't it be as simple as adding a new overload for byLine?
• Adam D. Ruppe (9/9) Mar 16 2014 If byLine returned a scope item, then it wouldn't be allowed to
• Marco Leise (6/9) Mar 16 2014 Yes, pointers to working buffers are a frequent use case for
• H. S. Teoh (8/17) Mar 16 2014 [...]
• Nick Treleaven (4/8) Mar 17 2014 It would certainly help when using foreach. I think there are cases
• Vladimir Panteleev (8/15) Mar 16 2014 For the record, if you want to keep all lines in memory anyway,
• Jakob Ovrum (3/21) Mar 16 2014 Reading all at once is also a problem for really big files.
• David Eagen (1/1) Mar 16 2014 copyByLine
• Vladimir Panteleev (5/28) Mar 16 2014 It is no different from:
• bearophile (5/8) Mar 16 2014 Once dup/idup become free functions in object you can also write:
• ponce (3/11) Mar 16 2014 f.lines
• monarch_dodra (23/41) Mar 16 2014 I'm for it in the sense that : "f.byLine.map!(x => x.idup)" is
• Timon Gehr (2/6) Mar 16 2014 What is it inconsistent with?
• monarch_dodra (7/15) Mar 16 2014 Nevermind. I thought you could write "map!dup" like you can with
• Joseph Rushton Wakeling (2/8) Mar 16 2014 byLineCopy ... ?
• Joseph Rushton Wakeling (3/4) Mar 16 2014 Sorry, didn't get far enough through the thread to see other people had ...
• bearophile (6/8) Mar 16 2014 Suggesting it again is useful, because it means more people find
• w0rp (14/22) Mar 16 2014 byLineCopy seems good to me. It *may* be an idea to change the
• Dmitry Olshansky (4/6) Mar 16 2014 f.lines?
• bearophile (8/9) Mar 17 2014 There is already a lines in std.stdio (but I don't use it much),
• Regan Heath (9/16) Mar 18 2014 Does this do the same as byLine or does it dup the lines?
• Etienne Cimon (3/10) Mar 16 2014 string[] split(File f, char[] sep = "\n"[, size_t max = 0]);
• Etienne (8/9) Mar 17 2014 Moving it to a template and rethinking the naming
• Joseph Cassman (5/13) Mar 16 2014 `f.lines` mentioned in a couple other posts seems straightforward
• safety0ff (4/4) Mar 17 2014 Just brainstorming:
• Mason McGill (2/6) Mar 17 2014
• Etienne (3/10) Mar 17 2014 .explode()
• H. S. Teoh (7/23) Mar 17 2014 Ew. If I wanted to go the PHP route, I'd go for .detonate() instead --
• John Stahara (3/5) Mar 17 2014 std.stdio.File.Shrapnel, obviously.
• Etienne Cimon (2/6) Mar 17 2014 ExplosionResult bomberMan = f.explode!(PewPew)
• Regan Heath (8/14) Mar 18 2014 Why not simply "lines".
• Steven Schveighoffer (5/11) Mar 18 2014 byImmutableLines
• Dicebot (3/18) Mar 18 2014 byPersistLines ?
• Regan Heath (9/27) Mar 18 2014 Why this fixation on "by"?
• Dicebot (5/11) Mar 18 2014 range vs container. I expect file.lines to be separate fully
• Regan Heath (6/18) Mar 19 2014 So "by" has come to signify "range" then?
• Andrei Alexandrescu (3/4) Mar 18 2014 First one I like.
• Marco Leise (14/20) Mar 19 2014 Also consider the consistency. The function does byLine() + a
• Danny Arends (4/12) Mar 19 2014 asString(s)
• Peter Alexander (3/6) Mar 19 2014 f.readLines
• Meta (3/9) Mar 19 2014 What about a simpler f.iter() or f.lineIter()?
• Andrei Alexandrescu (2/9) Mar 19 2014 Ideally it would be a variation of byLine. -- Andrei
• monarch_dodra (15/28) Mar 20 2014 What about simply templatizing (*) it?
• Chris (8/21) May 21 2014 byLineAsString
• bearophile (4/11) May 21 2014 The accepted name was byLineCopy :-)
• Chris (3/17) May 21 2014 Ok, that's a good name. I couldn't read the whole thread.

Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:

A classic idiom for reading lines and keeping them is f.byLine.map!(x => 
x.idup) to get strings instead of the buffer etc.

The current behavior trips new users on occasion, and the idiom solving 
it is very frequent. So what the heck - let's put that in a function, 
expose and document it nicely, and call it a day.

A good name would help a lot. Let's paint that bikeshed!


Andrei

"Jakob Ovrum" <jakobovrum gmail.com> writes:

On Sunday, 16 March 2014 at 16:58:36 UTC, Andrei Alexandrescu 
wrote:
 A classic idiom for reading lines and keeping them is 
 f.byLine.map!(x => x.idup) to get strings instead of the buffer 
 etc.

 The current behavior trips new users on occasion, and the idiom 
 solving it is very frequent. So what the heck - let's put that 
 in a function, expose and document it nicely, and call it a day.

 A good name would help a lot. Let's paint that bikeshed!


 Andrei

Maybe `byUniqueLine`.

We might also consider a higher order range (say, `byUnique`, in 
the same vein) that GC-clones elements in some generic fashion, 
then showing `f.byLine().byUnique()` front and centre in 
`byLine`'s documentation.

Just a couple of half-assed ideas to get the ball rolling.

Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:

On 3/16/14, 10:28 AM, Jakob Ovrum wrote:
 On Sunday, 16 March 2014 at 16:58:36 UTC, Andrei Alexandrescu wrote:
 A classic idiom for reading lines and keeping them is f.byLine.map!(x
 => x.idup) to get strings instead of the buffer etc.

 The current behavior trips new users on occasion, and the idiom
 solving it is very frequent. So what the heck - let's put that in a
 function, expose and document it nicely, and call it a day.

 A good name would help a lot. Let's paint that bikeshed!


 Andrei

 Maybe `byUniqueLine`.

No, looks like uniq.

Andrei

"sclytrack" <sclytrack spring.com> writes:

On Sunday, 16 March 2014 at 17:28:46 UTC, Jakob Ovrum wrote:
 On Sunday, 16 March 2014 at 16:58:36 UTC, Andrei Alexandrescu 
 wrote:
 A classic idiom for reading lines and keeping them is 
 f.byLine.map!(x => x.idup) to get strings instead of the 
 buffer etc.

 The current behavior trips new users on occasion, and the 
 idiom solving it is very frequent. So what the heck - let's 
 put that in a function, expose and document it nicely, and 
 call it a day.

 A good name would help a lot. Let's paint that bikeshed!


 Andrei

 Maybe `byUniqueLine`.

 We might also consider a higher order range (say, `byUnique`, 
 in the same vein) that GC-clones elements in some generic 
 fashion, then showing `f.byLine().byUnique()` front and centre 
 in `byLine`'s documentation.

 Just a couple of half-assed ideas to get the ball rolling.


   f.byLineIdup();

"Jakob Ovrum" <jakobovrum gmail.com> writes:

On Sunday, 16 March 2014 at 17:50:41 UTC, sclytrack wrote:
   f.byLineIdup();

If we can't find a descriptive name, what exactly are we buying 
over the status quo? Seems like the definition of a trivia 
function.

Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:

On 3/16/14, 10:52 AM, Jakob Ovrum wrote:
 On Sunday, 16 March 2014 at 17:50:41 UTC, sclytrack wrote:
   f.byLineIdup();

 If we can't find a descriptive name, what exactly are we buying over the
 status quo? Seems like the definition of a trivia function.

A function to send people to, instead of explaining an idiom.

Andrei

Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:

On 3/16/14, 10:52 AM, Jakob Ovrum wrote:
 Seems like the definition of a trivia function.

Trivial + frequent = good

see assert, enforce, ~, and many others.

Andrei

"Jakob Ovrum" <jakobovrum gmail.com> writes:

On Sunday, 16 March 2014 at 17:53:58 UTC, Andrei Alexandrescu 
wrote:
 On 3/16/14, 10:52 AM, Jakob Ovrum wrote:
 Seems like the definition of a trivia function.

 Trivial + frequent = good

 see assert, enforce, ~, and many others.

 Andrei

Fine. I'm guilty of still being in the camp that doesn't think a 
copying `byLine` is strictly needed, but I understand the 
pragmatism at play.

"bearophile" <bearophileHUGS lycos.com> writes:

Andrei Alexandrescu:

 A classic idiom for reading lines and keeping them is 
 f.byLine.map!(x => x.idup) to get strings instead of the buffer 
 etc.

This is essentially this issue, I will reopen it if you want:
https://d.puremagic.com/issues/show_bug.cgi?id=4474


 The current behavior trips new users on occasion,

In D the default behaviors should be "not tripping". And the 
optimized behavour should be on explicit request.

So I think byLine should dup on default, and it should not dup on 
request. As I explained in Issue 4474.

Perhaps this breaking change (I asked in Issue 4474) in byLine 
can't happen now.


 and the idiom solving it is very frequent. So what the heck - 
 let's put that in a function, expose and document it nicely, 
 and call it a day.

 A good name would help a lot. Let's paint that bikeshed!

A good function name for the copying version is:

"byDupLines"


An alternative solution is the opposite of that I was suggesting 
in Issue 4474:

byLine => not dup

byLine!true => copies every line with dup

Bye,
bearophile.

Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:

On 3/16/14, 10:49 AM, bearophile wrote:
 So I think byLine should dup on default, and it should not dup on
 request.

That will not happen.

Andrei

Ary Borenszweig <ary esperanto.org.ar> writes:

On 3/16/14, 2:50 PM, Andrei Alexandrescu wrote:
 On 3/16/14, 10:49 AM, bearophile wrote:
 So I think byLine should dup on default, and it should not dup on
 request.

 That will not happen.

 Andrei

Why not?

Don't optimize upfront. If later you find out that was the bottleneck 
then you change byLine to something more efficient and that's it. And 
new users don't get unexpected results.

I'm with bearophile here.

Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:

On 3/16/14, 12:28 PM, Ary Borenszweig wrote:
 On 3/16/14, 2:50 PM, Andrei Alexandrescu wrote:
 On 3/16/14, 10:49 AM, bearophile wrote:
 So I think byLine should dup on default, and it should not dup on
 request.

 That will not happen.

 Andrei

 Why not?

Performance regression.

Andrei

"Jakob Ovrum" <jakobovrum gmail.com> writes:

On Sunday, 16 March 2014 at 19:45:21 UTC, Andrei Alexandrescu 
wrote:
 On 3/16/14, 12:28 PM, Ary Borenszweig wrote:
 On 3/16/14, 2:50 PM, Andrei Alexandrescu wrote:
 On 3/16/14, 10:49 AM, bearophile wrote:
 So I think byLine should dup on default, and it should not 
 dup on
 request.

 That will not happen.

 Andrei

 Why not?

 Performance regression.

 Andrei

Not just that, but it would be a breaking change if we make the 
element type `string` instead of `char[]`, and if we *don't* do 
that, we end up in a poor compromise where it's neither optimally 
usable nor performant.

Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:

On 3/16/14, 10:49 AM, bearophile wrote:
 A good function name for the copying version is:

 "byDupLines"

It introduces the notion of "dup" to newbies. I'd rather go with a 
natural name.

Andrei

"w0rp" <devw0rp gmail.com> writes:

byLineWithCopy?
byCopiedLine?
byLineWithGarbage?

It's a hard thing to name.

"bearophile" <bearophileHUGS lycos.com> writes:

Andrei Alexandrescu:

 "byDupLines"

 It introduces the notion of "dup" to newbies. I'd rather go 
 with a natural name.

In Python you write:

for line in file("text.txt"):
     ...


So can't you make File iterable?

foreach (line; "text.txt".File) {}

Bye,
bearophile

"Jakob Ovrum" <jakobovrum gmail.com> writes:

On Sunday, 16 March 2014 at 17:58:45 UTC, bearophile wrote:
 In Python you write:

 for line in file("text.txt"):
     ...


 So can't you make File iterable?

 foreach (line; "text.txt".File) {}

 Bye,
 bearophile

Is `line` a byte, a character, or a line?

"bearophile" <bearophileHUGS lycos.com> writes:

Jakob Ovrum:

 Is `line` a byte, a character, or a line?

It's a line, as in all Python programs :-)

Note Jakob Ovrum that Andrei is asking for something for newbies. 
Otherwise for all other D programmers byDupLine is fine.

Bye,
bearophile

"HeiHon" <heihon example.com> writes:

On Sunday, 16 March 2014 at 17:51:31 UTC, Andrei Alexandrescu
wrote:
 On 3/16/14, 10:49 AM, bearophile wrote:
 "byDupLines"

 It introduces the notion of "dup" to newbies. I'd rather go 
 with a natural name.

If the newbie already knows that there is a difference between
byLine and byLineWhatever doesn't s/he already know about the
concept of dup?

Anyway
byLineCopy or
byLineCopied
sounds natural for me (native german).

Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:

On 3/16/14, 11:20 AM, HeiHon wrote:
 On Sunday, 16 March 2014 at 17:51:31 UTC, Andrei Alexandrescu
 wrote:
 On 3/16/14, 10:49 AM, bearophile wrote:
 "byDupLines"

 It introduces the notion of "dup" to newbies. I'd rather go with a
 natural name.

 If the newbie already knows that there is a difference between
 byLine and byLineWhatever doesn't s/he already know about the
 concept of dup?

Depends on the situation. Consider a D tutorial. It would feature little 
programs like "copy a file" or "put each line in a hashtable" etc.

 Anyway
 byLineCopy or
 byLineCopied
 sounds natural for me (native german).

Thanks.


Andrei

"bearophile" <bearophileHUGS lycos.com> writes:

HeiHon:

 byLineCopy or
 byLineCopied
 sounds natural for me (native german).


byLineCopy sounds good.

Bye,
bearophile

Marco Leise <Marco.Leise gmx.de> writes:

Am Sun, 16 Mar 2014 18:31:59 +0000
schrieb "bearophile" <bearophileHUGS lycos.com>:

 HeiHon:
 
 byLineCopy or
 byLineCopied
 sounds natural for me (native german).


 
 byLineCopy sounds good.
 
 Bye,
 bearophile

+1. Same reasoning as HeiHon gave.

-- 
Marco

Nick Treleaven <ntrel-public yahoo.co.uk> writes:

On 17/03/2014 03:58, Marco Leise wrote:
 Am Sun, 16 Mar 2014 18:31:59 +0000
 schrieb "bearophile" <bearophileHUGS lycos.com>:

 HeiHon:

 byLineCopy or
 byLineCopied
 sounds natural for me (native german).


 byLineCopy sounds good.

 Bye,
 bearophile

 +1. Same reasoning as HeiHon gave.

Implemented and reviewed by monarchdodra. Ready to merge (but more 
feedback would help):
https://github.com/D-Programming-Language/phobos/pull/2077

"bearophile" <bearophileHUGS lycos.com> writes:

Nick Treleaven:

 Implemented and reviewed by monarchdodra. Ready to merge (but 
 more feedback would help):
 https://github.com/D-Programming-Language/phobos/pull/2077

Good. Reopened:
https://issues.dlang.org/show_bug.cgi?id=4474

Bye,
bearophile

"HeiHon" <HeiHon example.com> writes:

 Depends on the situation. Consider a D tutorial. It would 
 feature little programs like "copy a file" or "put each line in 
 a hashtable" etc.

 Anyway
 byLineCopy or
 byLineCopied
 sounds natural for me (native german).


Another idea:
byLineDuplicate

Might be a bit (too) long, but probably could help leading to the 
topic of dup/idup and the like.

"Meta" <jared771 gmail.com> writes:

On Sunday, 16 March 2014 at 17:49:56 UTC, bearophile wrote:
 Andrei Alexandrescu:

 A classic idiom for reading lines and keeping them is 
 f.byLine.map!(x => x.idup) to get strings instead of the 
 buffer etc.

 This is essentially this issue, I will reopen it if you want:
 https://d.puremagic.com/issues/show_bug.cgi?id=4474


 The current behavior trips new users on occasion,

 In D the default behaviors should be "not tripping". And the 
 optimized behavour should be on explicit request.

 So I think byLine should dup on default, and it should not dup 
 on request. As I explained in Issue 4474.

 Perhaps this breaking change (I asked in Issue 4474) in byLine 
 can't happen now.


 and the idiom solving it is very frequent. So what the heck - 
 let's put that in a function, expose and document it nicely, 
 and call it a day.

 A good name would help a lot. Let's paint that bikeshed!

 A good function name for the copying version is:

 "byDupLines"


 An alternative solution is the opposite of that I was 
 suggesting in Issue 4474:

 byLine => not dup

 byLine!true => copies every line with dup

 Bye,
 bearophile.

Can't it be as simple as adding a new overload for byLine?

auto byLine(Terminator = char, Char = char, Flag!"cacheLines" 
cacheLines = Yes.cacheLines)(KeepTerminator keepTerminator = 
KeepTerminator.no, Terminator terminator = '\x0a')

Otherwise, we should probably just merge MonarchDodra's cache 
range adapter, and could even add a specialization for byLine's 
ByLine struct.

"Adam D. Ruppe" <destructionator gmail.com> writes:

If byLine returned a scope item, then it wouldn't be allowed to 
escape and throw a nice compile error pointing people in the 
right direction.... scope rox so much i want it so badly. (i 
guess here the mutability pretty much tells the story too tho)


But going with the idea that the duplicated one can be 
escaped/stored:

f.byLineStorable

or something like that. The name tells you that it is cool to 
store each line.

Marco Leise <Marco.Leise gmx.de> writes:

Am Sun, 16 Mar 2014 18:03:09 +0000
schrieb "Adam D. Ruppe" <destructionator gmail.com>:

 If byLine returned a scope item, then it wouldn't be allowed to 
 escape and throw a nice compile error pointing people in the 
 right direction.... scope rox so much i want it so badly.

Yes, pointers to working buffers are a frequent use case for
"borrowed pointers".

-- 
Marco

"H. S. Teoh" <hsteoh quickfur.ath.cx> writes:

On Mon, Mar 17, 2014 at 05:10:00AM +0100, Marco Leise wrote:
 Am Sun, 16 Mar 2014 18:03:09 +0000
 schrieb "Adam D. Ruppe" <destructionator gmail.com>:
 
 If byLine returned a scope item, then it wouldn't be allowed to 
 escape and throw a nice compile error pointing people in the 
 right direction.... scope rox so much i want it so badly.

 
 Yes, pointers to working buffers are a frequent use case for
 "borrowed pointers".

[...]

Just out of curiosity: what's holding up the implementation of scope?
It's been a long while since everybody has been clamoring for scope;
isn't it about time we got our heads together to find a solution?


T

-- 
All problems are easy in retrospect.

Nick Treleaven <ntrel-public yahoo.co.uk> writes:

On 16/03/2014 18:03, Adam D. Ruppe wrote:
 If byLine returned a scope item, then it wouldn't be allowed to escape
 and throw a nice compile error pointing people in the right
 direction.... scope rox so much i want it so badly. (i guess here the
 mutability pretty much tells the story too tho)

It would certainly help when using foreach. I think there are cases 
without foreach which would still be allowed to access previous range 
fronts though, as they could still be in scope.

"Vladimir Panteleev" <vladimir thecybershadow.net> writes:

On Sunday, 16 March 2014 at 16:58:36 UTC, Andrei Alexandrescu 
wrote:
 A classic idiom for reading lines and keeping them is 
 f.byLine.map!(x => x.idup) to get strings instead of the buffer 
 etc.

 The current behavior trips new users on occasion, and the idiom 
 solving it is very frequent. So what the heck - let's put that 
 in a function, expose and document it nicely, and call it a day.

 A good name would help a lot. Let's paint that bikeshed!

For the record, if you want to keep all lines in memory anyway, 
it's more efficient to just read the whole file at once then 
split it with splitLines(), because you avoid doing one memory 
allocation per line. The downside is if you want to keep only 
some of the lines on the heap in a long-running program - with 
this approach, the slices pin the entire file content.

"Jakob Ovrum" <jakobovrum gmail.com> writes:

On Sunday, 16 March 2014 at 18:06:00 UTC, Vladimir Panteleev 
wrote:
 On Sunday, 16 March 2014 at 16:58:36 UTC, Andrei Alexandrescu 
 wrote:
 A classic idiom for reading lines and keeping them is 
 f.byLine.map!(x => x.idup) to get strings instead of the 
 buffer etc.

 The current behavior trips new users on occasion, and the 
 idiom solving it is very frequent. So what the heck - let's 
 put that in a function, expose and document it nicely, and 
 call it a day.

 A good name would help a lot. Let's paint that bikeshed!

 For the record, if you want to keep all lines in memory anyway, 
 it's more efficient to just read the whole file at once then 
 split it with splitLines(), because you avoid doing one memory 
 allocation per line. The downside is if you want to keep only 
 some of the lines on the heap in a long-running program - with 
 this approach, the slices pin the entire file content.

Reading all at once is also a problem for really big files.

"David Eagen" <davideagen mailinator.com> writes:

copyByLine

"Vladimir Panteleev" <vladimir thecybershadow.net> writes:

On Sunday, 16 March 2014 at 18:14:18 UTC, Jakob Ovrum wrote:
 On Sunday, 16 March 2014 at 18:06:00 UTC, Vladimir Panteleev 
 wrote:
 On Sunday, 16 March 2014 at 16:58:36 UTC, Andrei Alexandrescu 
 wrote:
 A classic idiom for reading lines and keeping them is 
 f.byLine.map!(x => x.idup) to get strings instead of the 
 buffer etc.

 The current behavior trips new users on occasion, and the 
 idiom solving it is very frequent. So what the heck - let's 
 put that in a function, expose and document it nicely, and 
 call it a day.

 A good name would help a lot. Let's paint that bikeshed!

 For the record, if you want to keep all lines in memory 
 anyway, it's more efficient to just read the whole file at 
 once then split it with splitLines(), because you avoid doing 
 one memory allocation per line. The downside is if you want to 
 keep only some of the lines on the heap in a long-running 
 program - with this approach, the slices pin the entire file 
 content.

 Reading all at once is also a problem for really big files.

It is no different from:

f.byLine.map!(x => x.idup).array

...which is why I said "if you want to keep all lines in memory 
anyway".

"bearophile" <bearophileHUGS lycos.com> writes:

Andrei Alexandrescu:

 A classic idiom for reading lines and keeping them is 
 f.byLine.map!(x => x.idup) to get strings instead of the buffer 
 etc.

Once dup/idup become free functions in object you can also write:

f.byLine.map!idup

Bye,
bearophile

"ponce" <contact gam3sfrommars.fr> writes:

On Sunday, 16 March 2014 at 16:58:36 UTC, Andrei Alexandrescu 
wrote:
 A classic idiom for reading lines and keeping them is 
 f.byLine.map!(x => x.idup) to get strings instead of the buffer 
 etc.

 The current behavior trips new users on occasion, and the idiom 
 solving it is very frequent. So what the heck - let's put that 
 in a function, expose and document it nicely, and call it a day.

 A good name would help a lot. Let's paint that bikeshed!


 Andrei

f.lines

"monarch_dodra" <monarchdodra gmail.com> writes:

On Sunday, 16 March 2014 at 16:58:36 UTC, Andrei Alexandrescu 
wrote:
 A classic idiom for reading lines and keeping them is 
 f.byLine.map!(x => x.idup) to get strings instead of the buffer 
 etc.

 The current behavior trips new users on occasion, and the idiom 
 solving it is very frequent. So what the heck - let's put that 
 in a function, expose and document it nicely, and call it a day.

 A good name would help a lot. Let's paint that bikeshed!


 Andrei

I'm for it in the sense that : "f.byLine.map!(x => x.idup)" is 
*WRONG*.

It'll allocate on *every* call to front. Pipe it into a filter, 
and you have massive gratuitous memory allocations.

A named function will do better than the sloppy code above. So 
there's my vote...

...or... we could merge my "cache" proposal 
(https://github.com/D-Programming-Language/phobos/pull/1364). 
I'll admit I wrote it with *that* particular case in mind. Then, 
we promote:

"f.byLine.map!(x => x.idup).cache()"

But wait! Now it correct, but *definitly* not nooby friendly. So 
yes, my vote is to have a named function.

On Sunday, 16 March 2014 at 17:51:31 UTC, Andrei Alexandrescu 
wrote:
 On 3/16/14, 10:49 AM, bearophile wrote:
 A good function name for the copying version is:

 "byDupLines"

 It introduces the notion of "dup" to newbies. I'd rather go 
 with a natural name.

 Andrei

There comes a point where you have to learn the language to use 
it. "dup" is an array built-in; it's not ridiculous that expect 
the user to know it.

On Sunday, 16 March 2014 at 18:19:01 UTC, bearophile wrote:
 Once dup/idup become free functions in object you can also 
 write:

 f.byLine.map!idup

Indeed, I've been wanting to write this before. IMO, not being 
able to write it is a serious inconsistency.

Timon Gehr <timon.gehr gmx.ch> writes:

On 03/16/2014 08:27 PM, monarch_dodra wrote:
 f.byLine.map!idup

 Indeed, I've been wanting to write this before. IMO, not being able to
 write it is a serious inconsistency.

What is it inconsistent with?

"monarch_dodra" <monarchdodra gmail.com> writes:

On Sunday, 16 March 2014 at 20:27:35 UTC, Timon Gehr wrote:
 On 03/16/2014 08:27 PM, monarch_dodra wrote:
 f.byLine.map!idup

 Indeed, I've been wanting to write this before. IMO, not being 
 able to
 write it is a serious inconsistency.

 What is it inconsistent with?

Nevermind. I thought you could write "map!dup" like you can with 
most other *functions*, but "dup" is a member property function. 
In particular, things like "length"/"reserve" or whatnot are 
actually free functions that take an argument, but not dup.

So UFCS tends to blur the line between both, and what can be used 
where.

Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:

On 16/03/14 17:58, Andrei Alexandrescu wrote:
 A classic idiom for reading lines and keeping them is f.byLine.map!(x =>
x.idup)
 to get strings instead of the buffer etc.

 The current behavior trips new users on occasion, and the idiom solving it is
 very frequent. So what the heck - let's put that in a function, expose and
 document it nicely, and call it a day.

 A good name would help a lot. Let's paint that bikeshed!

byLineCopy ... ?

Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:

On 16/03/14 21:45, Joseph Rushton Wakeling wrote:
 byLineCopy ... ?

Sorry, didn't get far enough through the thread to see other people had already 
suggested this one.

"bearophile" <bearophileHUGS lycos.com> writes:

Joseph Rushton Wakeling:

 Sorry, didn't get far enough through the thread to see other 
 people had already suggested this one.

Suggesting it again is useful, because it means more people find 
that name sufficiently intuitive. So perhaps "byLineCopy" could 
be the winner name.

Bye,
bearophile

"w0rp" <devw0rp gmail.com> writes:

On Sunday, 16 March 2014 at 21:16:24 UTC, bearophile wrote:
 Joseph Rushton Wakeling:

 Sorry, didn't get far enough through the thread to see other 
 people had already suggested this one.

 Suggesting it again is useful, because it means more people 
 find that name sufficiently intuitive. So perhaps "byLineCopy" 
 could be the winner name.

 Bye,
 bearophile

byLineCopy seems good to me. It *may* be an idea to change the 
name of byLine very slowly via a long deprecation process so 
newcomers to the language don't start accidentally grabbing the 
shorter more obvious choice and run into problems when they are 
surprised by the semantics. byLineRef or some kind of thing. Just 
ignore me if that's a bad suggestion. I just kind of have this 
symmetry where if I have something named say "doFooSafely" I 
don't call the unsafe one "doFoo," I call it "doFooDangerously" 
or something obvious. Not that byLine is particularly "unsafe."

I considered calling a method named "disableGC" in DQt 
"createMemoryLeaks," but I decided that was pushing it a little 
too far. (It's a weird thing, but you actually need to use it 
some times. At least it's  system.)

Dmitry Olshansky <dmitry.olsh gmail.com> writes:

16-Mar-2014 20:58, Andrei Alexandrescu пишет:
 A classic idiom for reading lines and keeping them is f.byLine.map!(x =>
 x.idup) to get strings instead of the buffer etc.

f.lines?



-- 
Dmitry Olshansky

"bearophile" <bearophileHUGS lycos.com> writes:

Dmitry Olshansky:

 f.lines?

There is already a lines in std.stdio (but I don't use it much), 
search for:

foreach (string line; lines(stdin))

Here:
http://dlang.org/phobos/std_stdio.html

Bye,
bearophile

"Regan Heath" <regan netmail.co.nz> writes:

On Mon, 17 Mar 2014 12:38:23 -0000, bearophile <bearophileHUGS lycos.com>  
wrote:

 Dmitry Olshansky:

 f.lines?

 There is already a lines in std.stdio (but I don't use it much), search  
 for:

 foreach (string line; lines(stdin))

 Here:
 http://dlang.org/phobos/std_stdio.html

Does this do the same as byLine or does it dup the lines?

Can we replace or scrap it?


foreach(string line; f.lines)

is just too nice not to strive for, IMO.


R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

Etienne Cimon <etcimon gmail.com> writes:

On 2014-03-16 12:58, Andrei Alexandrescu wrote:
 A classic idiom for reading lines and keeping them is f.byLine.map!(x =>
 x.idup) to get strings instead of the buffer etc.

 The current behavior trips new users on occasion, and the idiom solving
 it is very frequent. So what the heck - let's put that in a function,
 expose and document it nicely, and call it a day.

 A good name would help a lot. Let's paint that bikeshed!


 Andrei

string[] split(File f, char[] sep = "\n"[, size_t max = 0]);

auto ln = f.split("\n", 15); // returns a gc allocated array of 15 lines

Etienne <etcimon gmail.com> writes:

On 2014-03-16 10:31 PM, Etienne Cimon wrote:
 auto ln = f.split("\n", 15); // returns a gc allocated array of 15 lines

Moving it to a template and rethinking the naming

auto lines = f.splitLn!15; // returns 15 lines
auto lines = f.split!";" // splits the file using ; sep
auto lines = f.splitLn

I consider this to be an essential part of file reading, not necessarily 
line separation but moving the file pointer forward a number of elements 
with a choice of characters.

"Joseph Cassman" <jc7919 outlook.com> writes:

On Sunday, 16 March 2014 at 16:58:36 UTC, Andrei Alexandrescu 
wrote:
 A classic idiom for reading lines and keeping them is 
 f.byLine.map!(x => x.idup) to get strings instead of the buffer 
 etc.

 The current behavior trips new users on occasion, and the idiom 
 solving it is very frequent. So what the heck - let's put that 
 in a function, expose and document it nicely, and call it a day.

 A good name would help a lot. Let's paint that bikeshed!


 Andrei

`f.lines` mentioned in a couple other posts seems straightforward 
and easy to remember.

Joseph

"safety0ff" <safety0ff.dev gmail.com> writes:

Just brainstorming:

f.splitByLine() or f.splitLines()

byLine -> process the file line by line
splitByLine -> split the file into lines

"Mason McGill" <mmcgill caltech.edu> writes:

f.copyLines // A bit less awkward than "f.byLinesCopy" IMO.

On Monday, 17 March 2014 at 07:13:49 UTC, safety0ff wrote:
 Just brainstorming:

 f.splitByLine() or f.splitLines()

 byLine -> process the file line by line
 splitByLine -> split the file into lines

Etienne <etcimon gmail.com> writes:

On 2014-03-16 12:58 PM, Andrei Alexandrescu wrote:
 A classic idiom for reading lines and keeping them is f.byLine.map!(x =>
 x.idup) to get strings instead of the buffer etc.

 The current behavior trips new users on occasion, and the idiom solving
 it is very frequent. So what the heck - let's put that in a function,
 expose and document it nicely, and call it a day.

 A good name would help a lot. Let's paint that bikeshed!


 Andrei

.explode()

i saw it in PHP ;)

"H. S. Teoh" <hsteoh quickfur.ath.cx> writes:

On Mon, Mar 17, 2014 at 03:52:48PM -0400, Etienne wrote:
 On 2014-03-16 12:58 PM, Andrei Alexandrescu wrote:
A classic idiom for reading lines and keeping them is f.byLine.map!(x
=> x.idup) to get strings instead of the buffer etc.

The current behavior trips new users on occasion, and the idiom
solving it is very frequent. So what the heck - let's put that in a
function, expose and document it nicely, and call it a day.

A good name would help a lot. Let's paint that bikeshed!


Andrei

 
 .explode()
 
 i saw it in PHP ;)

Ew. If I wanted to go the PHP route, I'd go for .detonate() instead --
at least there's a D in there! :-P

But what type would .explode() return? std.stdio.File.Explosion? :-P


T

-- 
It is the quality rather than the quantity that matters. -- Lucius Annaeus
Seneca

"John Stahara" <john.stahara+dlang gmail.com> writes:

On Monday, 17 March 2014 at 20:04:09 UTC, H. S. Teoh wrote:
 But what type would .explode() return? 
 std.stdio.File.Explosion? :-P

std.stdio.File.Shrapnel, obviously.

--jjs

Etienne Cimon <etcimon gmail.com> writes:

On 2014-03-17 16:06, John Stahara wrote:
 On Monday, 17 March 2014 at 20:04:09 UTC, H. S. Teoh wrote:
 But what type would .explode() return? std.stdio.File.Explosion? :-P

 std.stdio.File.Shrapnel, obviously.

 --jjs

ExplosionResult bomberMan = f.explode!(PewPew)

"Regan Heath" <regan netmail.co.nz> writes:

On Sun, 16 Mar 2014 16:58:38 -0000, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 A classic idiom for reading lines and keeping them is f.byLine.map!(x =>  
 x.idup) to get strings instead of the buffer etc.

 The current behavior trips new users on occasion, and the idiom solving  
 it is very frequent. So what the heck - let's put that in a function,  
 expose and document it nicely, and call it a day.

 A good name would help a lot. Let's paint that bikeshed!

Why not simply "lines".

foreach (line; file.lines)
   ...

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

"Steven Schveighoffer" <schveiguy yahoo.com> writes:

On Sun, 16 Mar 2014 12:58:38 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 A classic idiom for reading lines and keeping them is f.byLine.map!(x =>  
 x.idup) to get strings instead of the buffer etc.

 The current behavior trips new users on occasion, and the idiom solving  
 it is very frequent. So what the heck - let's put that in a function,  
 expose and document it nicely, and call it a day.

 A good name would help a lot. Let's paint that bikeshed!

byImmutableLines
byStringLines

-Steve

"Dicebot" <public dicebot.lv> writes:

On Tuesday, 18 March 2014 at 13:49:45 UTC, Steven Schveighoffer 
wrote:
 On Sun, 16 Mar 2014 12:58:38 -0400, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> wrote:

 A classic idiom for reading lines and keeping them is 
 f.byLine.map!(x => x.idup) to get strings instead of the 
 buffer etc.

 The current behavior trips new users on occasion, and the 
 idiom solving it is very frequent. So what the heck - let's 
 put that in a function, expose and document it nicely, and 
 call it a day.

 A good name would help a lot. Let's paint that bikeshed!

 byImmutableLines
 byStringLines

 -Steve

byPersistLines ?

"Regan Heath" <regan netmail.co.nz> writes:

On Tue, 18 Mar 2014 14:09:05 -0000, Dicebot <public dicebot.lv> wrote:

 On Tuesday, 18 March 2014 at 13:49:45 UTC, Steven Schveighoffer wrote:
 On Sun, 16 Mar 2014 12:58:38 -0400, Andrei Alexandrescu  
 <SeeWebsiteForEmail erdani.org> wrote:

 A classic idiom for reading lines and keeping them is f.byLine.map!(x  
 => x.idup) to get strings instead of the buffer etc.

 The current behavior trips new users on occasion, and the idiom  
 solving it is very frequent. So what the heck - let's put that in a  
 function, expose and document it nicely, and call it a day.

 A good name would help a lot. Let's paint that bikeshed!

 byImmutableLines
 byStringLines

 -Steve

 byPersistLines ?

Why this fixation on "by"?

lines
allLines
eachLine
everyLine

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

"Dicebot" <public dicebot.lv> writes:

On Tuesday, 18 March 2014 at 14:57:30 UTC, Regan Heath wrote:
 Why this fixation on "by"?

 lines
 allLines
 eachLine
 everyLine

 R

range vs container. I expect file.lines to be separate fully 
allocated entity that can be assigned and stored. file.byLines 
implies iteration without any guarantees about collection as a 
whole.

"Regan Heath" <regan netmail.co.nz> writes:

On Tue, 18 Mar 2014 15:03:16 -0000, Dicebot <public dicebot.lv> wrote:

 On Tuesday, 18 March 2014 at 14:57:30 UTC, Regan Heath wrote:
 Why this fixation on "by"?

 lines
 allLines
 eachLine
 everyLine

 R

 range vs container. I expect file.lines to be separate fully allocated  
 entity that can be assigned and stored. file.byLines implies iteration  
 without any guarantees about collection as a whole.

So "by" has come to signify "range" then?

"eachLine" does not imply a container but an iteration/range..

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:

On 3/18/14, 6:49 AM, Steven Schveighoffer wrote:
 byStringLines

First one I like.

Andrei

Marco Leise <Marco.Leise gmx.de> writes:

Am Tue, 18 Mar 2014 09:49:32 -0700
schrieb Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>:

 On 3/18/14, 6:49 AM, Steven Schveighoffer wrote:
 byStringLines

 
 First one I like.
 
 Andrei

Also consider the consistency. The function does byLine() + a
copy, right? Not something entirely different.

byLine
byLine[Copy/Dup/...]

would also show up in order on lists like Phobos documentation
or IDE completion lists, so it cannot be missed. Which is
important, because the obvious choice has the unexpected
buffer reuse.

Consistency, consistency, consistency. That's how Java got
big. How PHP made it, I don't know.

-- 
Marco

"Danny Arends" <Danny.Arends gmail.com> writes:

On Wednesday, 19 March 2014 at 13:44:35 UTC, Marco Leise wrote:
 Am Tue, 18 Mar 2014 09:49:32 -0700
 schrieb Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>:

 On 3/18/14, 6:49 AM, Steven Schveighoffer wrote:
 byStringLines

 
 First one I like.
 
 Andrei


asString(s)
byString(s)

my 2 cents

"Peter Alexander" <peter.alexander.au gmail.com> writes:

On Sunday, 16 March 2014 at 16:58:36 UTC, Andrei Alexandrescu 
wrote:
 A classic idiom for reading lines and keeping them is 
 f.byLine.map!(x => x.idup) to get strings instead of the buffer 
 etc.

f.readLines

"Meta" <jared771 gmail.com> writes:

On Wednesday, 19 March 2014 at 22:30:55 UTC, Peter Alexander 
wrote:
 On Sunday, 16 March 2014 at 16:58:36 UTC, Andrei Alexandrescu 
 wrote:
 A classic idiom for reading lines and keeping them is 
 f.byLine.map!(x => x.idup) to get strings instead of the 
 buffer etc.

 f.readLines

What about a simpler f.iter() or f.lineIter()?

Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:

On 3/19/14, 4:53 PM, Meta wrote:
 On Wednesday, 19 March 2014 at 22:30:55 UTC, Peter Alexander wrote:
 On Sunday, 16 March 2014 at 16:58:36 UTC, Andrei Alexandrescu wrote:
 A classic idiom for reading lines and keeping them is f.byLine.map!(x
 => x.idup) to get strings instead of the buffer etc.

 f.readLines

 What about a simpler f.iter() or f.lineIter()?

Ideally it would be a variation of byLine. -- Andrei

"monarch_dodra" <monarchdodra gmail.com> writes:

On Thursday, 20 March 2014 at 01:38:38 UTC, Andrei Alexandrescu 
wrote:
 On 3/19/14, 4:53 PM, Meta wrote:
 On Wednesday, 19 March 2014 at 22:30:55 UTC, Peter Alexander 
 wrote:
 On Sunday, 16 March 2014 at 16:58:36 UTC, Andrei Alexandrescu 
 wrote:
 A classic idiom for reading lines and keeping them is 
 f.byLine.map!(x
 => x.idup) to get strings instead of the buffer etc.

 f.readLines

 What about a simpler f.iter() or f.lineIter()?

 Ideally it would be a variation of byLine. -- Andrei

What about simply templatizing (*) it?

byLine() => implicit char[]
byLine!string() => dupes

This has the double advantage that:
* Does not introduce a new symbol
* Can be customized for wchar/dchar

In particular, the wchar thing could be of interest to those that 
are reading a file they now is UTF16/UCS2.

(*) Technically, it's already a template, but the parameters are 
inferred. We could make it so that we can use explicitly defined 
parameters. In particular, so that the output type is specified 
before the terminator type. AFAIK, the transition can be made 
seemlessly with no breakage.

"Chris" <wendlec tcd.ie> writes:

On Thursday, 20 March 2014 at 01:38:38 UTC, Andrei Alexandrescu 
wrote:
 On 3/19/14, 4:53 PM, Meta wrote:
 On Wednesday, 19 March 2014 at 22:30:55 UTC, Peter Alexander 
 wrote:
 On Sunday, 16 March 2014 at 16:58:36 UTC, Andrei Alexandrescu 
 wrote:
 A classic idiom for reading lines and keeping them is 
 f.byLine.map!(x
 => x.idup) to get strings instead of the buffer etc.

 f.readLines

 What about a simpler f.iter() or f.lineIter()?

 Ideally it would be a variation of byLine. -- Andrei

byLineAsString

In my own APIs I usually use "AsString" in the signature to make 
it clear what you'll get, if an entity can take many forms (an 
array of strings, user defined types). In other cases toString().

byLineToString

?

"bearophile" <bearophileHUGS lycos.com> writes:

Chris:

 byLineAsString

 In my own APIs I usually use "AsString" in the signature to 
 make it clear what you'll get, if an entity can take many forms 
 (an array of strings, user defined types). In other cases 
 toString().

 byLineToString

 ?

The accepted name was byLineCopy :-)

Bye,
bearophile

"Chris" <wendlec tcd.ie> writes:

On Wednesday, 21 May 2014 at 13:46:21 UTC, bearophile wrote:
 Chris:

 byLineAsString

 In my own APIs I usually use "AsString" in the signature to 
 make it clear what you'll get, if an entity can take many 
 forms (an array of strings, user defined types). In other 
 cases toString().

 byLineToString

 ?

 The accepted name was byLineCopy :-)

 Bye,
 bearophile

Ok, that's a good name. I couldn't read the whole thread. 
Sometimes I can't keep up with all the messages.