• d-bugmail puremagic.com (74/74) Mar 28 2018 https://issues.dlang.org/show_bug.cgi?id=18692

d-bugmail puremagic.com writes:

https://issues.dlang.org/show_bug.cgi?id=18692

          Issue ID: 18692
           Summary: assignment of std.regex.Captures reads freed memory
                    from 2.072.0 to 2.078.3 inclusive
           Product: D
           Version: D2
          Hardware: x86_64
                OS: Linux
            Status: NEW
          Severity: regression
          Priority: P1
         Component: phobos
          Assignee: nobody puremagic.com
          Reporter: martin.dorey hitachivantara.com

When I was working on:

Issue 18691: assigning a std.regex.Captures with 3 or more groups causes double
free

... I couldn't understand how this change:

https://github.com/dlang/phobos/commit/872673d5570f1ee79df4b9e47d8f3d2cf4e49536

... the "Use ref-counting for Captures struct" from  DmitryOlshansky, could be
right.  How can he get away with having the _refcount that applies to
big_matches not stored in an object with the same lifetime as big_matches?  I
have about two days' familiarity with D, so I assume I'm missing something. 
That said, here's a program that goes wrong with the first release of dmd that
included that change.  First, it working on the preceding version.

martind swiftboat:~/tmp/D134366$ cat two_years.d
import std.regex;

void main() {
    auto rx = regex("()()()");
    auto ma = "".matchFirst(rx);
    auto ma2 = ma;
    ma = ma2;
    ma[1];
}
martind swiftboat:~/tmp/D134366$ dmd --version
DMD64 D Compiler v2.071.0
Copyright (c) 1999-2015 by Digital Mars written by Walter Bright
martind swiftboat:~/tmp/D134366$ dmd two_years.d && MALLOC_PERTURB_=1
./two_years 
martind swiftboat:~/tmp/D134366$ 

2.072.0 was afflicted by
https://digitalmars.com/d/archives/digitalmars/D/dmd_or_phobos_were_broken_in_ubuntu_16.10_d-apt_294148.html,
hence the extra shared library parameters, which make no difference on 2.071.0.

martind balance:~/tmp/D134366$ dmd --version
DMD64 D Compiler v2.072.0
Copyright (c) 1999-2016 by Digital Mars written by Walter Bright
martind balance:~/tmp/D134366$ dmd -defaultlib=libphobos2.so -fPIC two_years.d
&& MALLOC_PERTURB_=1 ./two_years 
core.exception.RangeError std/regex/package.d(565): Range violation
----------------
??:? _d_arraybounds [0xf693d091]
??:? std.regex.__array [0xf68f6f1e]
??:? std.regex.Captures!(immutable(char)[],
ulong).Captures.opIndex!().opIndexinout(pure nothrow  trusted
inout(immutable(char)[]) function(ulong)) [0xb4f37081]
??:? _Dmain [0xb4f0f60f]
??:? _D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ6runAllMFZ9__lambda1MFZv
[0xf696d68f]
??:? void rt.dmain2._d_run_main(int, char**, extern (C) int
function(char[][])*).tryExec(scope void delegate()) [0xf696d5bb]
??:? void rt.dmain2._d_run_main(int, char**, extern (C) int
function(char[][])*).runAll() [0xf696d634]
??:? void rt.dmain2._d_run_main(int, char**, extern (C) int
function(char[][])*).tryExec(scope void delegate()) [0xf696d5bb]
??:? _d_run_main [0xf696d51f]
??:? main [0xb4f3a67f]
??:? __libc_start_main [0xf55f52b0]
martind balance:~/tmp/D134366$ 

valgrind is unhappy too and more verbosely, making it explicit that the problem
is reading from freed memory.  The problems continue up to and including
2.078.3 but disappear for 2.079.0 and are absent in master.  I suspect that the
 MartinNowak change I finger in Issue 18691 fixed this particular reference
counting issue at the expense of introducing that one.

--