.NET Framework - Why does C# not support explicit abstract interface implementations?

Asked By Marcel_Müller on 23-Mar-11 08:37 AM
The following code is not valid - why?

interface ITest
{ string Get();
}

abstract class ATest : ITest
{ abstract string ITest.Get(); // Invalid
}

class Test : ATest
{ string ITest.Get()
{ return "Buh!";
}
}

The usual work around is

abstract class ATest : ITest
{ protected abstract string Get();
string ITest.Get()
{ return Foo();
}
}

class Test : ATest
{ protected override string Get()
{ return "Buh!";
}
}

Unfortunately this has the following drawbacks:
- The implementation is no longer explicit. It must be at least protected.
- You have to work around for name clashes manually, e.g if the class
has itself a Get method with another signature or if two implemented
interfaces have a method with the same name but different signature or
semantic.
- Calling ITest.Foo() causes a second vtable lookup. This creates a
completely unnecessary runtime overhead. I am unsure whether the JIT can
compensate for that.


Another work around is to throw an exception in the abstract base class
nut never execute this code:

abstract class ATest : ITest
{ string ITest.Get()
{ throw NotImplementedException();
}
}

class Test : ATest, ITest
{ string ITest.Get()
{ return "Buh!";
}
}

But this has some other serious drawbacks.
- The compiler can no longer detect if class Test implements the
interface completely.
- The derived class Test must repeat all explicit interface
implementations of ITest not only the (logically) abstract ones.


Marcel




Peter Duniho replied to Marcel_Müller on 23-Mar-11 10:29 AM
Explicit implementations are private.  You cannot override private
members, but an abstract member must be overridden at some point.

The two are simply mutually exclusive.  It would not make any sense to
allow private abstract members of any kind, and that includes explicit
interface implementations.

Pete
Peter Duniho replied to Marcel_Müller on 23-Mar-11 10:44 AM
Also???


I do not even know what you mean here.  The accessibility of the member
should not AFAIK affect the organization of an object's vtable (explicit
or not, the vtable has to support interface member dispatch from only
the interface type), nor is there actually any requirement that a vtable
even be used for virtual member dispatch for that matter.  The CLR is
free to implement JIT-compiled virtual members in any way that preserves
the defined semantics.

For example, if in a particular section of code the JIT compiler can be
sure that the interface implementation at hand is always from a specific
type, the method call may not be done via vtable or similar mechanism at
all.

And for that matter, IMHO an explicit interface implementation only
makes it _less_ likely such optimizations could be made, because it
increases the chances that in the code where the call to an interface
member is made, the original type of the object has been lost as far as
the static compilation is concerned (since the object reference needed
to be cast to the interface type itself at some point).

That said, even if your statement is correct and there is an additional
dereference required for the non-private interface dispatch as compared
to private (explicit) interface dispatch, it is highly unlikely that
extra dereference is going to make any real difference in the overall
performance of the code.  The cost of dereferencing memory is utterly
inconsequential as compared to other things such as actually _calling_ a
function and actually _doing work_ in said function.

If and when you have a performance problem related to non-private
abstract interface implementations, that is a matter to address with the
usual code optimization techniques.  Until then, it is not something to
be concerned with.
Marcel Müller replied to Peter Duniho on 23-Mar-11 11:28 AM
Well, that is somehow a point of view question. I never intended to
/override/ an interface implementation, since there is nothing to
override so far.

But you are right, abstract functions operate in the same way. You must
override them to implement them. I always found this little intuitive,
since from the technical point of view implementing an interface
function and implementing an abstract function is the same.


Other languages have no problems with that.


Marcel
Marcel Müller replied to Peter Duniho on 23-Mar-11 11:44 AM
Yes, but the IL-Code for an explicit or public interface implementation
has one callvirt statement, while the proxy function that maps the
explicit implementation to the abstract method contains another callvirt
statement. This is obvious, because a derived class could reimplement
the interface and override the abstract function independently (but this
is not intended). So I guess the second access to the vtable cannot be
optimized away unless the JIT knows that the class is sealed.


That's it. And since explicit interface implementations can only be
called after a cast to the interface type, the type information is
almost always lost.


However, this is likely true in most cases.
Especially because the first callvirt is sufficient to prevent any
inline expansion of the function.


I personally dislike constraints that make life more complicated to the
programmer as well as to the CPU. That's all. And I think the feature
would not require any changes to the IL.


Marcel
Peter Duniho replied to Marcel Müller on 23-Mar-11 09:26 PM
You misunderstand what I am saying.  The act of implementing an interface
is exactly the same as overriding a member.  Abstract members are
implemented with the "override" keyword, and as far as the compiler and
run-time are concerned, it is the same thing.

In fact, an interface is really nothing more than an abstract class with
no implementation.  it is just that C# provides a somewhat more
partitioned way of looking at it.

The fact that you are not required to provide the "override" keyword when
implementing interface members is simply a convenience C# offers.  The
underlying mechanism is still the same.


Some do.  Some do not.  it is simply a matter of choices during language
design.  No one way is best.

If you prefer to be able to have private virtual members, then C# is not
the language for you.  Go use something else.

Pete
Peter Duniho replied to Marcel Müller on 24-Mar-11 11:49 PM
I have been puzzling over your post for a day now, and I still cannot
figure out what you mean.

There is no proxy function involved.  When a class declares itself as
implementing an interface, that class's vtable is configured to include
entries for that interface's implementation.  If the class is later
inherited and the same interface is reimplemented, the new
implementation members are simply inserted into the vtable slot where
the base class members were.

One of the weirder things (well, IMHO) that might come up is a base
class with an abstract or virtual implicit implementation for an
interface, where a sub-class overrides that implicit implementation, and
at the same time explicitly re-implements the interface.  In that case,
the compiler generates two vtable slots: one for the interface
implementation and one for the base class virtual method.  That allows
the explicit implementation to be called any time the interface type is
used for the call, and the virtual (now non-interface) implementation to
be called any time the base or derived class type is used for the call.

In that case, instances of the base class refer to a vtable that uses
only the original virtual method implementing the interface, while
instances of the derived class use a vtable that supports the parallel
implementations (i.e. the base class virtual member, as well as the
interface member).  In either case, the call to the method (whichever
one) is still just a simple, single virtual call.

For example:

interface IA
{
void Method();
}

class A : IA
{
public abstract void Method();
}

class B : A, IA
{
public override void Method() { }
void IA.Method() { }
}

Then:

A a = new B();

((IA)a).Method(); // 1: calls IA.Method()
((IA)(B)a).Method(); // 2: calls IA.Method()
((B)a).Method(); // 3: calls Method()
a.Method(); // 4: calls Method()

There is no need for some sort of proxy method.  The call just uses a
different slot from the vtable.  (Given the first line "1:", the second
line "2:" should not be surprising???after all, it is the same object; I
just included it to emphasize that what is important is the compile-time
type of the object, not how you got to that compile-type type).


I still do not understand what "second vtable lookup" you are talking
about here.  At worst, an explicit implementation of an interface has
the same overhead as calling any other virtual member where the call
site cannot be optimized to the exact method (which is usually the case).


me that whatever the cost of the virtual call, it is inconsequential as
compared to real work being done?  Or something else?


There is only one call.


More complicated to the CPU?  I do not see how.  As for the programmer,
it seems to me that it is more complicated to _allow_ explicit abstract
interface implementations.  It adds a new specialized scenario to the
compiler, the language specification, and of course to the programmer's
necessary knowledge.

I suppose you might argue that _all_ virtual members should be allowed
to be private, but I would not agree with that position either.  I think
it is a serious deficiency of language design to allow derived types to
have any real access to private members at all.  If you want to allow
sub-classes to override a member, do not make it private.  "private"
means no one outside of the class knows about it; it makes no sense to
me for a language to allow one to override something one knows nothing
about.

Designing a programming language is not just about what the IL allows.
it is about providing a good, organized, well-structured language that
leads a programmer to writing good, maintainable, correct code.

Pete
Marcel Müller replied to Peter Duniho on 25-Mar-11 11:57 AM
Yes, but not in the example I provided.

interface ITest
{ string Get();
}

abstract class ATest : ITest
{ protected abstract string Get();
string ITest.Get()
{ return Foo();
}
}

class Test : ATest
{ protected override string Get()
{ return "Buh!";
}
}

Test vtable:
- ITest.Get()  ->  ATest.Get()
- Get()        ->  Test.Get()

Calling ITest.Get() on an object of type Test will invoke ATest.Get()
and that will lookup Get() and invoke Test.Get().


In contrast:

interface ITest
{ string Get();
}

abstract class ATest : ITest
{ public abstract string Get();
}

class Test : ATest
{ public override string Get()
{ return "Buh!";
}
}

Test vtable:
- ITest  ->  ITest vtable:
- ITest.Get()  ->  Test.Get()
- Get()        ->  Test.Get()

Calling ITest.Get() on an object of type Test will directly invoke
Test.Get().


The only option is, that the JIT compiler identifies that the method
ATest.Get() is semantically equivalent with Test.Get() and rewrites the
vtable to the one like in the second example.



I agree. An vtable call is not that expensive.



No, see above.

The first example compiles to

.class private abstract auto ansi beforefieldinit ATest
extends [mscorlib]System.Object
implements Test.ITest
{
.method public hidebysig newslot abstract virtual instance string
Get() cil managed
{
}

.method private hidebysig newslot virtual final instance string
Test.ITest.Get() cil managed
{
.override Test.ITest::Get
.maxstack 8
ldarg.0
callvirt instance string Test.ATest::Get()
ret
}
}

.class private auto ansi beforefieldinit Test
extends Test.ATest
{
.method public hidebysig virtual instance string Get() cil managed
{
.maxstack 8
ldstr "Buh!"
ret
}
}

The first call occurs when calling ITest.Get(), the second one in the
implementation of Test.ITest::Get().

And the .NET 3.5 JIT complier also does not optimize it on x86.

itest.Get();
00000132  mov         ecx,dword ptr [ebp-60h]  <-- here is the first
00000135  call        dword ptr ds:[00940A90h] <-- vtable lookup
0000013b  nop

string ITest.Get()
{	return Get();
00000000  push        ebp
00000001  mov         ebp,esp
00000003  sub         esp,8
00000006  mov         dword ptr [ebp-4],ecx
00000009  cmp         dword ptr ds:[04000FA4h],0
00000010  je          00000017
00000012  call        76AE4741
00000017  mov         ecx,dword ptr [ebp-4]
0000001a  mov         eax,dword ptr [ecx]      <-- here is the second
0000001c  call        dword ptr [eax+38h]      <-- vtable lookup
0000001f  mov         dword ptr [ebp-8],eax
00000022  mov         eax,dword ptr [ebp-8]
00000025  mov         esp,ebp
00000027  pop         ebp
00000028  ret



I agree with you. The private abstract functions itself are of no
particular use.
One /could/ use it to mark a function as not intended to be called by
derived classes. But making it private does not hit the nail on the head
either, because when implementing it in a derived class it will become
private to the derived class which effectively bypasses the access
restriction.



D'accord!

But this also implies to enable the programmer to express as many
constraints as possible with a reasonable amount of work. These
constraints are checked at compile time rather than at runtime or maybe
by the user. This significantly improves code quality and/or reduces the
test amount. To require reflection does not achieve this objective.

(That's why I miss const references in .NET. Accidentally modifying an
immutable, cached object, shared by many threads could produce undefined
behavior bugs that take years to find. Working with deep const proxies
on the other side causes a significant overhead at development time and
at run time.)

Of course, introducing generics in .NET 2.0 was a far step in that
direction. It made many down casts obsolete and reduced code redundancy.


Marcel
Peter Duniho replied to Marcel Müller on 25-Mar-11 10:52 PM
No, it does not.  Calling ITest.Get() on an object of type Test will
invoke ATest.Get(), _only_.  The method Get() in the class Test is not
called.

The class ATest winds up with two different vtable slots for Get(), and
the compiler decides statically which slot to use based on the
compile-time type of the object.

When the compile-time type is ITest, the ITest.Get() slot is used and
that slot contains the explicit interface method ITest.Get() in the
class ATest.  When the compile-time type is Test, the ATest.Get() slot
is used and that slot contains the override of the abstract ATest.Get()
method.


The JIT compiler just uses whatever the C# compiler tells it to, based
on the vtable slots allocated during C# compilation.  The JIT compiler
may optimize things, of course.  But it has to start with the vtable the
C# compiler gives it.  And that vtable does not involve any proxy
methods.  The only methods in the vtable are the ones declared in the
types the vtable applies to.


Yes, see above.


The above does not correspond to the code you posted.  The code you
posted calls an undeclared method named "Foo()" in the explicit
interface implementation ITest.Get() in the Test class.

But let us assume for the moment that the code you posted had a call to
not the compiler putting the second call in.  That's _you_ doing it.  If
you do not want it, do not put it there.  Simple.

Don't blame poor design and implementation on the compiler.  There is
nothing about the language that forces that choice.  It is foolish at
best to claim that not allowing abstract explicit interface
implementations necessarily leads to a two-step call, disingenuous at worst.

In any case, since on the one hand you agree that one should not be able
to override private members, I do not see you can still claim that C#
should allow abstract explicit interface implementations.  The two are
mutually exclusive.


Why should it?  It hardly seems like something that would have any sort
of benefit, nor that would even come up often enough to warrant adding
the complexity to the compiler.


It depends on what you are doing.  In specific scenarios, IMHO it is much
better to use the type-safe delegate/anonymous method approach.  On the
other hand, reflection is useful when you are writing code to handle
types your own code has not seen before, nor could see in advance.

In the latter case, I do not see any way around it.  The types being
handled are not available at compile-time, so there is no way to be
type-safe in any case.

Pete