Difference between revisions of "DIP49"

From D Wiki
Jump to: navigation, search
(Add a section "Why use 'inout' keyword for 'unique' postblit?")
 
(3 intermediate revisions by the same user not shown)
Line 7: Line 7:
 
|-
 
|-
 
|Version:
 
|Version:
|1
+
|2
 
|-
 
|-
 
|Status:
 
|Status:
Line 16: Line 16:
 
|-
 
|-
 
|Last Modified:
 
|Last Modified:
|2013-11-10
+
|2013-12-17
 
|-
 
|-
 
|Author:
 
|Author:
 
|Hara Kenji
 
|Hara Kenji
 +
|-
 +
|Links:
 +
|[http://forum.dlang.org/post/mailman.326.1384066006.9546.digitalmars-d@puremagic.com Related forum discussion]
 
|}
 
|}
  
 
== Abstract ==
 
== Abstract ==
  
If an object has some indirections, postblit should handle them correctly.
+
This DIP resolves the postblit design issues.
This DIP will classify postblits into four types, and will resolve the postblit design issues.
+
 
 +
== Motivation ==
 +
 
 +
Plain old postblit concept has an issue that postblits does nothing for the type qualifiers reinterpretation of the object indirections. For example:
 +
 
 +
struct X {
 +
    int[] arr;
 +
    this(this) {}
 +
}
 +
 
 +
If you want to copy mutable X to immutable, compiler automatically memcpy the object image before the postblit call. But the user-defined postblit does nothing for the 'arr' field. So, the int[] arr will be interpreted to immutable(int[]) by copy.
 +
 
 +
int[] arr = [1,2,3];
 +
X m = X(arr);
 +
immutable X i = m;  // IF this copy invoke X.this(this)
 +
static assert(is(typeof(i.arr) == immutable));
 +
assert(i.arr == [1,2,3]);
 +
arr[] = 100;
 +
assert(i.arr == [1,2,3]);  // fails!
 +
 
 +
Qualified postblits should provide ways to handle such reinterpretation.
  
 
== Description ==
 
== Description ==
 +
 +
In the ideal world, objects would have only one of the two qualifiers - mutable or <code>immutable</code>.
 +
So, mutable postblit (supports mutable to mutable copy), and immutable postblit (supports immutable to immutable copy) are necessary at least.
 +
 +
However, in actual D code, there are two wildcard qualifiers, <code>const</code> and <code>inout</code>.
 +
Therefore, we should have ways copying an object which the original qualifier is unknown.
 +
And, we would also need a way to copy objects to distinct qualifiers (mutable to immutable, etc).
 +
For those requests, this DIP will provide additional two concepts, inout postblit and const postblit.
  
 
=== Mutable Postblit ===
 
=== Mutable Postblit ===
  
Signature: <code>this(this);</code>
+
If a postblit is unqalified, it will be used for the copy from mutable source to mutable destination.
  
Modifying indirections via mutable fields is allowed.
+
  struct S {
 
+
     int num;
  struct SM {
 
     int var;
 
 
     int[] arr;
 
     int[] arr;
 
     <strong>this(this)</strong> {
 
     <strong>this(this)</strong> {
         static assert(typeof(this.var) == int);
+
         static assert(typeof(this.num) == int);
 
         static assert(typeof(this.arr) == int[]);
 
         static assert(typeof(this.arr) == int[]);
         var = 1;     // OK
+
 +
         num = 1;
 +
        // value fields can be initialized again
 +
 
         arr[] += 1;  // OK
 
         arr[] += 1;  // OK
 
     }
 
     }
 
  }
 
  }
  
The indirections may be shared with original objects, so mutable postblit may rewrite the representation of the source object.
+
Modifying indirections via mutable fields is allowed. The indirections may be shared with original objects, so mutable postblit may rewrite the representation of the source object.
  
  SM sm1 = SM(1, [1,2,3]);
+
  S sm1 = S(1, [1,2,3]);
  SM sm2 = sm1;  // mutable postblit is called
+
  S sm2 = sm1;  // mutable postblit is called
 
  assert(sm2.marr == [2,3,4]);
 
  assert(sm2.marr == [2,3,4]);
 
  assert(sm1.marr == [2,3,4]); // modified
 
  assert(sm1.marr == [2,3,4]); // modified
 
  assert(sm1.arr.ptr == sm2.arr.ptr);
 
  assert(sm1.arr.ptr == sm2.arr.ptr);
 
Mutable postblit will be invoked when the source object is mutable.
 
* mutable to mutable [m1]
 
* mutable to const [m2]
 
You can regard the [m2] case as that the generated mutable copy is referred by const reference.
 
 
=== Constant Postblit ===
 
 
Signature: <code>this(this) const;</code>
 
 
After blit copy is done, you cannot modify indirections because they are qualified at least with const.
 
On the other hand, you can re-initialize references,
 
 
struct SC {
 
    int var;
 
    int[] arr;
 
    <strong>this(this) const</strong> {
 
        static assert(typeof(this.var) == const int);
 
        static assert(typeof(this.arr) == const int[]);
 
        var = 1;    // OK
 
      //arr[] += 1;  // NG
 
        arr = this.arr.dup; // OK
 
    }
 
}
 
 
Inside constant postblit, indirections may be shared with the original object, so modifying them is disallowed.
 
On the other hand, you can re-initialize references.
 
 
Constant postblit will be invoked for the copies that qualifier transition is equal or weaken.
 
* mutable to mutable [c1]
 
* mutable to const [c2]
 
* const to const [c3]
 
* immutable to const [c4]
 
* immutable to immutable [c5]
 
  
 
=== Immutable Postblit ===
 
=== Immutable Postblit ===
  
Signature: <code>this(this) immutable;</code>
+
If a postblit is qalified with <code>immutable</code>, it will be used for the copy from immutable source to immutable destination.
  
Inside immutable postblit, you cannot modify indirections, because they are qualified with immutable.
+
  struct S {
 
+
     int num;
  struct SI {
 
     int var;
 
 
     int[] arr;
 
     int[] arr;
 
     <strong>this(this) immutable</strong> {
 
     <strong>this(this) immutable</strong> {
         static assert(typeof(this.var) == immutable int);
+
         static assert(typeof(this.num) == immutable int);
 
         static assert(typeof(this.arr) == immutable int[]);
 
         static assert(typeof(this.arr) == immutable int[]);
         var = 1;     // OK
+
      //arr[] += 1; // NG
+
         num = 1;
         arr = this.arr.idup; // OK
+
        // value fields can be initialized again
 +
 +
        //arr[] += 1;
 +
        // cannot modify immutable data
 +
 +
         arr = this.arr.idup;
 +
        // reference field rebinding is allowed
 
     }
 
     }
 
  }
 
  }
  
Immuatble postblit will be invoked when the source object is immutable.
+
Of course, you cannot modify indirections during postblitting, because they are qualified with immutable.
* immutable to const [i1]
 
* immutable to immutable [i2]
 
You can regard the [i2] case as that the generated immutable copy is referred by const reference.
 
  
==== Overload resolution (1) ====
+
=== Inout Postblit ===
  
These three postblits can be overloaded each other. At that time, following rule will be applied for overload resolution:
+
If a postblit is qalified with <code>inout</code>, it is used for the copy when source and destination have same qualifier.
  
;a. If both [m2] and [c2] are possible, [c2] is preferred.
+
struct S {
: [m2] will invoke 2-step operations - copy an mutable object to mutable, then qualify the copy by const.
+
    int num;
: On the other hand, [c2] will invoke 1-step operation - copy mutable object to const object.
+
    int[] marr;
: So, shorter distance operation c2 will be chosen.
+
    const int[] carr;
 +
    immutable int[] iarr;
 +
 +
    static int[] gmarr;
 +
    static immutable int[] giarr;
 +
 +
    <strong>this(this) inout</strong> {
 +
        num = 1;
 +
        // value fields can be initialized again
 +
 +
        //marr[] += 1;
 +
        // cannot modify indirections, because at least they are qualified with inout
 +
        static assert(is(typeof(marr) == inout int[]));
 +
        // you can keep reference fields as-is.
 +
 +
        //carr = garr;
 +
        // initializing const refereneces by mutable/const data is disallowed. Instead
 +
        carr = garr.dup;  // initialize by unique expression.
 +
        carr = giarr;    // or by immutable data.
 +
 +
        iarr = giarr;
 +
        // initializing immutable reference by immutable data is allowed.
 +
    }
 +
}
  
; b. If both [c4] and [i1] are possible, [c4] is preferred.
+
If inout postblit does nothing for reference fields, the source and destination may share indirections.
: [i2] will invoke 2-step operations - copy an immutable object to immutable, then qualify the copy by const.
 
: On the other hand, [c4] will invoke 1-step operation - copy immutable object to const object.
 
: So, shorter distance operation c4 will be chosen.
 
  
From the practical view, the rule has the advantage to be able to take copy-on-write strategy. Even if your struct type needs deep copy for mutable objects, by defining <code>this(this) const {}</code>, you can elide the cost when you need a const copy.
+
  struct S {
 
 
=== Unique Postblit ===
 
 
 
Signature: <code>this(this) inout;</code>
 
 
 
Above three postblits will extend the plain old postblit definition naturally, by adding a perspective about qualifier conversion in there.
 
However it is still insufficient for the copy operations of between incompatible qualifiers (eg. mutable to immutable, const to mutable, etc).
 
 
 
To fix the last issue, I propose "unique postblit" concept.
 
 
 
The main issue of plain old postblit concept is that postblit will do nothing for the type qualifiers reinterpretation of the object indirections.
 
For example:
 
 
 
  struct X {
 
 
     int[] arr;
 
     int[] arr;
     this(this) {}
+
     this(this) inout {
 +
        // do nothing for this.arr
 +
    }
 +
}
 +
void main() {
 +
    S s1 = S([1,2,3]);
 +
    S s2 = s1;  // inout postblit is called
 +
    assert(s1.arr.ptr == s2.arr.ptr);
 +
    s1.arr[] = 10;
 +
    assert(s2.arr == [10, 10, 10]);
 
  }
 
  }
  
If you want to copy mutable <code>X</code> to immutable, compiler automatically memcopy the object image before the postblit call. However the user-defined postblit will do nothing for the 'arr' field. Then, the int[] arr will be interpreted to immutable(int[]) by copy.
+
=== Const Postblit ===
  
int[] arr = [1,2,3];
+
If a postblit is qualified with <code>const</code>, it will be used to make arbitrary qualified copy from arbitrary qualified source.
X m = X(arr);
+
 
immutable X i = m;  // IF this copy invoke X.this(this)
+
Mutable and immutable postblit will extend the plain old postblit definition naturally,
static assert(is(typeof(i.arr) == immutable));
+
by adding a perspective about qualifier conversion in there.
assert(i.arr == [1,2,3]);
+
However it is still insufficient for the copy operations of between incompatible qualifiers
arr[] = 100;
+
(eg. mutable to immutable, const to mutable, etc).
assert(i.arr == [1,2,3]);  // fails!
 
  
If the postblit call can guarantee that the copied result won't share any indirections with others, the problem will be disappeared. In other words, compiler should provide a way to guarantee that a postblit will generate a "unique" copy from the source.
+
If a postblit call can guarantee that the copied object owns no reference to the external state,
For that postblit, I assign the signature <code>this(this) inout;</code>. And to satisfy the requirement, compiler will enforce following rule
+
the object may be convertible to arbitrary qualifier.
 +
In other words, the postblit would have an ability to construct "unique" copy from arbitrary qualified source object.
  
* Inside unique postblit, all of non-immutable indirections should be re-initialized by ''Unique Expression''s.
+
So, it could also be called "unique postblit", based on the concept.
  
For example:
+
Inside const postblit, compiler will enforce following rule:
 +
- all of non-immutable indirections must be re-initialized by Unique Expressions.
  
  struct SU {
+
  struct S {
     int var;
+
     int num;
 
     int[] arr;
 
     int[] arr;
     <strong>this(this) inout</strong> {
+
     <strong>this(this) const</strong> {
         static assert(typeof(this.var) == inout int);
+
         num = 1;
         static assert(typeof(this.arr) == inout int[]);
+
         // value fields can be initialized again
         // In each control flow paths:
+
        if (true) {
+
         //arr = arr;
            arr = arr; // rhs is not an unique expression, so compiler will reject this path.
+
        // rhs is not an unique expression, so compiler will reject this initialization.
         } else if (true) {
+
            ;          // also rejected if do nothing for the arr field
+
         // also forbidden to do nothing for the arr field
         }
+
            arr = arr.dup; // arr.dup makes unique expression, so compiler accepts this path.
+
         arr = arr.dup;
        }
+
        // arr.dup makes unique expression, so compiler accepts this line.
 
     }
 
     }
 
  }
 
  }
  
The definition of ''Unique Expressions'' is:
+
==== The definition of ''Unique Expressions'' ====
  
 
# Basic literal values (integers, complexes, characters)
 
# Basic literal values (integers, complexes, characters)
Line 184: Line 199:
 
#: If the literal has subsequent elements, the sub expressions should also be unique.
 
#: If the literal has subsequent elements, the sub expressions should also be unique.
 
# Expressions that has no indirections
 
# Expressions that has no indirections
#: For example, multiply integers returns integer rvalue, and integer has no indirections, so multiply expression will be unique.
+
#: For example, multiply integers returns rvalue integer, and integer has no indirections, so multiply expression will be unique.
 
#: <code>int a, b, c = a * b;  // the multiply will become unique expression</code>
 
#: <code>int a, b, c = a * b;  // the multiply will become unique expression</code>
# An unique object constructed by unique constructor
+
# An unique object constructed by const constructor
# An unique object constructed by unique postblit
+
# An unique object constructed by const postblit
 
# A field variable of unique object
 
# A field variable of unique object
 
#: <code>unique_obj.var</code> is also unique.
 
#: <code>unique_obj.var</code> is also unique.
Line 204: Line 219:
 
# New expression with unique arguments
 
# New expression with unique arguments
 
#: If a struct type is new-ed with literal syntax, same as "literal values" case.
 
#: If a struct type is new-ed with literal syntax, same as "literal values" case.
#: If a class type is new-ed, the called constructor should be unique constructor.
+
#: If a class type is new-ed, the called constructor should be const constructor.
  
 
(maybe this list is not complete)
 
(maybe this list is not complete)
  
Enforcing the rule will make the result of calling <code>this(this) inout</code> unique. And it will guarantee that the copy has no shared indirections. Therefore, implicit casting the copy to any qualifier is valid and won't break type system.
+
=== Overloading of qualified postblits ===
  
SU sm = S(1, [1,2,3]);
+
If mutable postblit is defined, it is alwasy used for the copies:
immutable SU si = sm;  // unique postblit is called
+
* mutable to mutable
    // In here, si.arr is always re-allocated by the unique postblit.
+
* mutable to const
assert(cast(void*)si.arr.ptr !is cast(void*)sm.arr.ptr);    // OK
+
  Note that: Different from the previous version of this DIP, mutable postblit is always used for mutable to const copy.
  
Note that, if an object indirection will be finally marked as immutable or const, initializing it by non-unique immutable data is allowed.
+
If immutable postblit is defined, it is alwasy used for the copies:
 
+
* immutable to const
immutable(int)[] global;
+
* immutable to immutable
struct S {
+
  Note that: Different from the previous version of this DIP, immutable postblit is always used for immutable to const copy.
    immutable(int)[] iarr;
 
    const(int)[] carr;
 
    this(this) inout {
 
        iarr = global;  // OK
 
        carr = global;  // OK
 
    }
 
}
 
  
==== Overload resolution (2) ====
+
If inout postblit is defined,
 +
* if mutable postblit is not defined, it will be used for the copies:
 +
** mutable to mutable
 +
** mutable to const
 +
* if immutable postblit is not defined, it will be used for the copies:
 +
** immutable to const
 +
** immutable to immutable
 +
* it is always used for other copies that qualifier transition is equal or weaken
 +
** const to const
 +
** inout to inout
 +
** inout to const
  
You may overload unique postblit with other three postblits. If so, it will have less priority than others.
+
If const postblit is defined,
 +
* it is always used for copies between incompatible qualifiers:
 +
** mutable to immutable
 +
** const to mutable
 +
** const to immutable
 +
** immutable to mutable
 +
** inout to mutable
 +
** inout to immutable
 +
* if other postblits are not defined, can cover corresponding copy directions.
 +
** mutable to mutable
 +
** mutable to const
 +
** const to const
 +
** immutable to const
 +
** immutable to immutable
 +
** inout to inout
 +
** inout to const
  
struct SX {
+
These priority order is defined based on the following rule:
    this(this) const { ... }
+
# If source is mutable or immutable, most specialized postblits (mutable/immutable postblit) will be  used, if they exists.
    this(this) inout { ... }
+
# If inout postblit exists and applicable, it is used.
}
+
# If const postblit exists, it is used.
SX sm;
+
# Otherwise, "cannot copy" error will occur.
SX sm2 = sm;            // constant postblit is called
 
const SX sc = sm;      // constant postblit is called
 
immutable SX si = sm;  // unique postblit is called
 
  
 
== Concatenation of field postblits ==
 
== Concatenation of field postblits ==
Line 258: Line 288:
 
  }
 
  }
 
  struct S2 {
 
  struct S2 {
     A a;
+
     immutable A a;
     immutableB b;
+
     B b;
     // a.this(this); is callable only for the copy S2 to S2.
+
     // a.this(this); is not callable for the copy from immutable A to immutable A.
     // b.this(this) immutable is callable only for the copy immutable(S2) to immutable(S2)
+
     // b.this(this) immutable is callable only for the copy from B to B
 
     // Therefore compiler cannot generate appropriate postblit implicitly for S2.
 
     // Therefore compiler cannot generate appropriate postblit implicitly for S2.
 
     // Then S2 will be marked as uncopyable.
 
     // Then S2 will be marked as uncopyable.
Line 269: Line 299:
  
 
  struct S3 {
 
  struct S3 {
     A a;
+
     immutable A a;
     immutableB b;
+
     B b;
     this(this) { // or const or immutable or inout, as you needed
+
     this(this) { // or immutable or inout or const, as you needed
 
         // When this postblit is invoked, Both a and b are immediately after the bitwise copy.
 
         // When this postblit is invoked, Both a and b are immediately after the bitwise copy.
 
         // So re-initializing both fields will be enforced by compiler.
 
         // So re-initializing both fields will be enforced by compiler.
         a = A();  // Re-initializing must be required
+
         a = immutable A();  // Re-initializing must be required
         b = B(); // Re-initializing must be required
+
         b = B();           // Re-initializing must be required
 
     }
 
     }
 
  }
 
  }
 +
 +
Rules to generate combined postblit from struct fields:
 +
# If all of the fields have const postblits, the enclosing struct can generate const postblit automatically.
 +
# If all of the fields have inout postblits, the enclosing struct can generate inout postblit automatically.
 +
# If all of the fields have immutable postblits, the enclosing struct can generate immutable postblit automatically.
 +
# If all of the fields have postblits which support the copy between same qualifiers, the enclosing struct can generate mutable postblit automatically.
  
 
== Fix for TypeInfo ==
 
== Fix for TypeInfo ==
Line 287: Line 323:
  
 
* <code>typeid(S).postblit(&obj)</code> will call "mutable postblit"
 
* <code>typeid(S).postblit(&obj)</code> will call "mutable postblit"
* <code>typeid(const S).postblit(&obj)</code> will call "constant postblit"
 
 
* <code>typeid(immutable S).postblit(&obj)</code> will call "immutable postblit"
 
* <code>typeid(immutable S).postblit(&obj)</code> will call "immutable postblit"
* <code>typeid(inout S).postblit(&obj)</code> will call "unique postblit"
+
* <code>typeid(inout S).postblit(&obj)</code> will call "inout postblit"
 +
* <code>typeid(const S).postblit(&obj)</code> will call "const postblit"
  
 
If S does not support corresponding postblit, <code>TypeInfo.postblit</code> will throw <code>Error</code> in runtime.
 
If S does not support corresponding postblit, <code>TypeInfo.postblit</code> will throw <code>Error</code> in runtime.
Line 296: Line 332:
 
     this(this) immutable;
 
     this(this) immutable;
 
  }
 
  }
  typeid(S).postblit(&obj);   // will throw Error
+
 +
// trying to invoke mutable postblit will throw Error
 +
  typeid(S).postblit(&obj);
  
 
== Impact to the existing code ==
 
== Impact to the existing code ==
Line 311: Line 349:
 
     S sm;
 
     S sm;
 
     immutable S si;
 
     immutable S si;
 +
 
     S sm2 = si;            // invoke S.this(this)
 
     S sm2 = si;            // invoke S.this(this)
 
     immutable S si2 = sm;  // invoke S.this(this)
 
     immutable S si2 = sm;  // invoke S.this(this)
Line 316: Line 355:
  
 
But after qualified postblit introduced, it won't work anymore.
 
But after qualified postblit introduced, it won't work anymore.
To fix the issue, you need to change the postblit signature to <code>this(this) inout</code>.
+
To fix the issue, you need to change the postblit signature to <code>this(this) const</code>.
  
Other changes will be '''undefined behavior''', because until now D language hadn't defined well about qualified postblits (Including <code>this(this) const;</code>).
+
Other changes will be '''undefined behavior''', because until now D language hadn't defined well about qualified postblits.
  
== Why use 'inout' keyword for 'unique' postblit? ==
+
== Why 'const' postblit will called to copy arbitrary qualified object? ==
  
Because, the unique postblit concept is strongly related to <code>inout</code> type qualifier.
+
When an object is constructed by <code>const</code> postblit, the destination object would have either mutable or immutable qualifier.
 
+
And, const method is always callable on both mutable and immutable object.
Consider a case that copying inout struct inside inout function.
 
  
 
  struct S {
 
  struct S {
    int[] arr;
+
     this(this) const { ... }
     this(this) ??? { }
 
 
  }
 
  }
  int[] foo(inout S src)
+
  void main() {
  {
+
    S sm;
     S dst = src; // copy inout S to S
+
    immutable S si;
     return dst.arr;
+
   
 +
    // const postblit is callable on constructing mutable object
 +
     S sm2 = sm; //   mutable to mutable
 +
    S sm3 = si;  // immutable to mutable
 +
 +
    // const postblit is callable on constructing mutable object
 +
    immutable S si2 = si;  // immutable to immutable
 +
     immutable S si3 = sm; //  mutable to immutable
 
  }
 
  }
  
If the struct S has postblit, what sholud be done for "copying S from inout to mutable"?
+
There's no mutation against widely known "const method" concept.
 
 
1. You cannot modify elements of arr field, because originally it may be immutable.
 
2. You must re-initialize arr field by unique expression, otherwise it may break type system
 
 
 
The requirements are exactly same as what necessary for unique postblit.
 
Even if you want to copy inout struct to immutable, there's no difference.
 
 
 
So therefore, "creating unique copy from arbitrary qualified source" is exactly same as "treating the copy source as inout" essentially.
 
  
 
== Rationale ==
 
== Rationale ==
  
The definition of ''Unique Constructor'' could be improved by using ''Unique Expression'' definition.
+
See also [[DIP53]].
  
 
== Copyright ==
 
== Copyright ==

Latest revision as of 01:08, 19 December 2013

Title: Define qualified postblit
DIP: 49
Version: 2
Status: Draft
Created: 2013-11-10
Last Modified: 2013-12-17
Author: Hara Kenji
Links: Related forum discussion

Abstract

This DIP resolves the postblit design issues.

Motivation

Plain old postblit concept has an issue that postblits does nothing for the type qualifiers reinterpretation of the object indirections. For example:

struct X {
    int[] arr;
    this(this) {}
}

If you want to copy mutable X to immutable, compiler automatically memcpy the object image before the postblit call. But the user-defined postblit does nothing for the 'arr' field. So, the int[] arr will be interpreted to immutable(int[]) by copy.

int[] arr = [1,2,3];
X m = X(arr);
immutable X i = m;  // IF this copy invoke X.this(this)
static assert(is(typeof(i.arr) == immutable));
assert(i.arr == [1,2,3]);
arr[] = 100;
assert(i.arr == [1,2,3]);  // fails!

Qualified postblits should provide ways to handle such reinterpretation.

Description

In the ideal world, objects would have only one of the two qualifiers - mutable or immutable. So, mutable postblit (supports mutable to mutable copy), and immutable postblit (supports immutable to immutable copy) are necessary at least.

However, in actual D code, there are two wildcard qualifiers, const and inout. Therefore, we should have ways copying an object which the original qualifier is unknown. And, we would also need a way to copy objects to distinct qualifiers (mutable to immutable, etc). For those requests, this DIP will provide additional two concepts, inout postblit and const postblit.

Mutable Postblit

If a postblit is unqalified, it will be used for the copy from mutable source to mutable destination.

struct S {
    int num;
    int[] arr;
    this(this) {
        static assert(typeof(this.num) == int);
        static assert(typeof(this.arr) == int[]);

        num = 1;
        // value fields can be initialized again

        arr[] += 1;  // OK
    }
}

Modifying indirections via mutable fields is allowed. The indirections may be shared with original objects, so mutable postblit may rewrite the representation of the source object.

S sm1 = S(1, [1,2,3]);
S sm2 = sm1;  // mutable postblit is called
assert(sm2.marr == [2,3,4]);
assert(sm1.marr == [2,3,4]); // modified
assert(sm1.arr.ptr == sm2.arr.ptr);

Immutable Postblit

If a postblit is qalified with immutable, it will be used for the copy from immutable source to immutable destination.

struct S {
    int num;
    int[] arr;
    this(this) immutable {
        static assert(typeof(this.num) == immutable int);
        static assert(typeof(this.arr) == immutable int[]);

        num = 1;
        // value fields can be initialized again

        //arr[] += 1;
        // cannot modify immutable data

        arr = this.arr.idup;
        // reference field rebinding is allowed
    }
}

Of course, you cannot modify indirections during postblitting, because they are qualified with immutable.

Inout Postblit

If a postblit is qalified with inout, it is used for the copy when source and destination have same qualifier.

struct S {
    int num;
    int[] marr;
    const int[] carr;
    immutable int[] iarr;

    static int[] gmarr;
    static immutable int[] giarr;

    this(this) inout {
        num = 1;
        // value fields can be initialized again

        //marr[] += 1;
        // cannot modify indirections, because at least they are qualified with inout
        static assert(is(typeof(marr) == inout int[]));
        // you can keep reference fields as-is.

        //carr = garr;
        // initializing const refereneces by mutable/const data is disallowed. Instead
        carr = garr.dup;  // initialize by unique expression.
        carr = giarr;     // or by immutable data.

        iarr = giarr;
        // initializing immutable reference by immutable data is allowed.
    }
}

If inout postblit does nothing for reference fields, the source and destination may share indirections.

struct S {
    int[] arr;
    this(this) inout {
        // do nothing for this.arr
    }
}
void main() {
    S s1 = S([1,2,3]);
    S s2 = s1;  // inout postblit is called
    assert(s1.arr.ptr == s2.arr.ptr);
    s1.arr[] = 10;
    assert(s2.arr == [10, 10, 10]);
}

Const Postblit

If a postblit is qualified with const, it will be used to make arbitrary qualified copy from arbitrary qualified source.

Mutable and immutable postblit will extend the plain old postblit definition naturally, by adding a perspective about qualifier conversion in there. However it is still insufficient for the copy operations of between incompatible qualifiers (eg. mutable to immutable, const to mutable, etc).

If a postblit call can guarantee that the copied object owns no reference to the external state, the object may be convertible to arbitrary qualifier. In other words, the postblit would have an ability to construct "unique" copy from arbitrary qualified source object.

So, it could also be called "unique postblit", based on the concept.

Inside const postblit, compiler will enforce following rule: - all of non-immutable indirections must be re-initialized by Unique Expressions.

struct S {
    int num;
    int[] arr;
    this(this) const {
        num = 1;
        // value fields can be initialized again

        //arr = arr;
        // rhs is not an unique expression, so compiler will reject this initialization.

        // also forbidden to do nothing for the arr field

        arr = arr.dup;
        // arr.dup makes unique expression, so compiler accepts this line.
    }
}

The definition of Unique Expressions

  1. Basic literal values (integers, complexes, characters)
  2. Complex literal values (struct literals, array literals, AA literals)
    If the literal has subsequent elements, the sub expressions should also be unique.
  3. Expressions that has no indirections
    For example, multiply integers returns rvalue integer, and integer has no indirections, so multiply expression will be unique.
    int a, b, c = a * b; // the multiply will become unique expression
  4. An unique object constructed by const constructor
  5. An unique object constructed by const postblit
  6. A field variable of unique object
    unique_obj.var is also unique.
  7. An address of unique object
    &unique_obj is also unique.
  8. A copy of an array
    iff the element type supports generating unique copy.
    • unique_array[n]
    • unique_array[n .. m]
  9. An element(s) of unique array
    • unique_array[n]
    • unique_array[n .. m]
  10. Concatenation of arrays
    By definition, concat expression will always create a newly allocated array. So iff the element type has no reference, the result will be unique.
  11. Pure function call which returns unique object
  12. New expression with unique arguments
    If a struct type is new-ed with literal syntax, same as "literal values" case.
    If a class type is new-ed, the called constructor should be const constructor.

(maybe this list is not complete)

Overloading of qualified postblits

If mutable postblit is defined, it is alwasy used for the copies:

  • mutable to mutable
  • mutable to const
 Note that: Different from the previous version of this DIP, mutable postblit is always used for mutable to const copy.

If immutable postblit is defined, it is alwasy used for the copies:

  • immutable to const
  • immutable to immutable
 Note that: Different from the previous version of this DIP, immutable postblit is always used for immutable to const copy.

If inout postblit is defined,

  • if mutable postblit is not defined, it will be used for the copies:
    • mutable to mutable
    • mutable to const
  • if immutable postblit is not defined, it will be used for the copies:
    • immutable to const
    • immutable to immutable
  • it is always used for other copies that qualifier transition is equal or weaken
    • const to const
    • inout to inout
    • inout to const

If const postblit is defined,

  • it is always used for copies between incompatible qualifiers:
    • mutable to immutable
    • const to mutable
    • const to immutable
    • immutable to mutable
    • inout to mutable
    • inout to immutable
  • if other postblits are not defined, can cover corresponding copy directions.
    • mutable to mutable
    • mutable to const
    • const to const
    • immutable to const
    • immutable to immutable
    • inout to inout
    • inout to const

These priority order is defined based on the following rule:

  1. If source is mutable or immutable, most specialized postblits (mutable/immutable postblit) will be used, if they exists.
  2. If inout postblit exists and applicable, it is used.
  3. If const postblit exists, it is used.
  4. Otherwise, "cannot copy" error will occur.

Concatenation of field postblits

If a struct has a field which has postblit, compiler will generate postblit implicitly for the enclosing struct.

struct A {
    this(this);
}
struct S1 {
    A a;
    // Compiler will generate this(this); implicitly
}

If struct fields have incompatible postblits, compiler implicitly mark the enclosing struct uncopyable.

struct B {
    this(this) immutable;
}
struct S2 {
    immutable A a;
    B b;
    // a.this(this); is not callable for the copy from immutable A to immutable A.
    // b.this(this) immutable is callable only for the copy from B to B
    // Therefore compiler cannot generate appropriate postblit implicitly for S2.
    // Then S2 will be marked as uncopyable.
}

To make S2 copyable, you need to define postblit by hand.

struct S3 {
    immutable A a;
    B b;
    this(this) { // or immutable or inout or const, as you needed
        // When this postblit is invoked, Both a and b are immediately after the bitwise copy.
        // So re-initializing both fields will be enforced by compiler.
        a = immutable A();  // Re-initializing must be required
        b = B();            // Re-initializing must be required
    }
}

Rules to generate combined postblit from struct fields:

  1. If all of the fields have const postblits, the enclosing struct can generate const postblit automatically.
  2. If all of the fields have inout postblits, the enclosing struct can generate inout postblit automatically.
  3. If all of the fields have immutable postblits, the enclosing struct can generate immutable postblit automatically.
  4. If all of the fields have postblits which support the copy between same qualifiers, the enclosing struct can generate mutable postblit automatically.

Fix for TypeInfo

TypeInfo.postblit(in void* p); is invoked on array copy/concatenation by druntime. So it must support qualified postblits. For that, following change is necessary.

If a struct S exists:

  • typeid(S).postblit(&obj) will call "mutable postblit"
  • typeid(immutable S).postblit(&obj) will call "immutable postblit"
  • typeid(inout S).postblit(&obj) will call "inout postblit"
  • typeid(const S).postblit(&obj) will call "const postblit"

If S does not support corresponding postblit, TypeInfo.postblit will throw Error in runtime.

struct S {
    this(this) immutable;
}

// trying to invoke mutable postblit will throw Error
typeid(S).postblit(&obj);

Impact to the existing code

Currently, if a struct has no indirection fields, the user-defined postblit will be invoked on incompatible qualifier copies unrelated to its qualifier.

struct S
{
    int value;  // has no indirection
    this(this) { printf("postblit\n"); }
}
void main()
{
    S sm;
    immutable S si;

    S sm2 = si;            // invoke S.this(this)
    immutable S si2 = sm;  // invoke S.this(this)
}

But after qualified postblit introduced, it won't work anymore. To fix the issue, you need to change the postblit signature to this(this) const.

Other changes will be undefined behavior, because until now D language hadn't defined well about qualified postblits.

Why 'const' postblit will called to copy arbitrary qualified object?

When an object is constructed by const postblit, the destination object would have either mutable or immutable qualifier. And, const method is always callable on both mutable and immutable object.

struct S {
    this(this) const { ... }
}
void main() {
    S sm;
    immutable S si;

    // const postblit is callable on constructing mutable object
    S sm2 = sm;  //   mutable to mutable
    S sm3 = si;  // immutable to mutable

    // const postblit is callable on constructing mutable object
    immutable S si2 = si;  // immutable to immutable
    immutable S si3 = sm;  //   mutable to immutable
}

There's no mutation against widely known "const method" concept.

Rationale

See also DIP53.

Copyright

This document has been placed in the Public Domain.