Com S 541 Lecture -*- Outline -*- * The Scala Type System ** Basic concepts *** type and type error ------------------------------------------ BASIC CONCEPTS OF TYPE AND CLASS Def: A *protocol* is a set of messages (method names and arguments) that can be sent to an object Def: A *signature* is a set of method names and argument types Some questions: - Who decides what set of objects that obeys a protocol is a type? - What is a type error? ------------------------------------------ ... the language? the programmer? if it's the programmer, then we can avoid confusing objects with the same protocol but different semantics so it would be an ADT this is essentially the "by name" vs "structural" type checking idea Def: A *structural type* is a description of a set of objects that can be sent messages in a given protocol Def: A *nominal type* is a name and an associated signature The same distinction applies to subtyping (i.e., does it correspond to behavioral subtyping?) Scala has by name typing, as we'll see ... assignment of value of type T to a variable of type S? so what? treating a T as a S? why do we care? to prevent type errors -- but that's circular to prevent message not understood errors -- ok Donahue: all objects are bit strings, want to make sure don't misinterpret them What's a misinterpretation? perhaps interpretation outside an implementation module *** behavioral reasoning aided by type checker Q: can we make even stronger behavioral guarantees? ------------------------------------------ REASONING ABOUT ADTs Data type induction argument example: Sets represented by lists To prove: every Set's list has no duplicates How to prove this? ------------------------------------------ show it holds when sets are created, and that if it holds for a set argument (e.g., self) then it holds for any mutated or created sets Q: What part of the program has to be examined for this to work? *** Seals (Morris 1973) way to describe invariant preservation want module to be able to seal data before passing it to world ------------------------------------------ SEAL GOALS Only by using the ADTs ops can a client: alter = Clients cannot: discover = impersonate = SEALS MECHANISM createSeal -> (seal_t, unseal_t) gives unique seal and unseal functions Properties: unseal_t(seal_t(x)) = x unseal_t(anything else) is an error ------------------------------------------ ... change state of an object ... access components of object ... fake an object of the ADT Q: How could these cause problems for a program that kept a table in sorted order? Q: Does Scala prevent impersonation? yes (unlike, say, Smalltalk) idea: limit use of seal_t to a small area of code Q: How does this allow for data type induction? want to prove for all x: P(unseal_t(x)) must show that for all y: if seal_t is applied to y, then P(y) holds. Q: How could we use seals to define what a "type error" means? attempt to access sealed data without proper capability each object sealed according to its type (seal_T). set of primitive operations have access to unseal_T Q: How does this relate to security in Scala or Java? *** static and conservative nature of type systems ------------------------------------------ TYPE SYSTEMS AS STATIC APPROXIMATIONS Does this code have a type error? var i: int = 0; var f: Foo = _; if (myReader.read_bool()) { i = i + 1; } else { f = i; } ------------------------------------------ Yes, because we might execute the else part since can't know for sure, assume the worse this is a conservative approximation Q: Why are all elements of an array assumed to have the same type? so don't have to know the run-time state Hence generally have typed containers (variables) - typed containers: container of type T may only point to T objects allows earlier catching of errors seals unnecessary for objects in container e.g., savings for arrays - typed objects only: containers untyped can only catch errors on access to contained object seals must be stored with object ------------------------------------------ BASIC STRATEGY OF STATIC TYPE SYSTEMS Variables have a statically declared type Expresssions are assigned a type based on Examples: ------------------------------------------ ... the types of their subexpressions, and the signatures of operators (typing rules), inductively ... i: Int; (i + 3 - 2) * 4 *** soundness and completeness ------------------------------------------ SOUNDNESS AND COMPLETENESS A type system is *sound* iff A type system is *complete* iff ------------------------------------------ ... whenever the type system says an expression has a type, and the expression can be evaluated to a value, then the evaluation does not produce a type error. ... whenever the evaluation does not produce a type error, the type system sas it has a type. Don't usually get completeness, since it's conservative. Q: How to prove this? prove something stronger: that if the type system says an expression E has type T, then the value of E has type T. This property is sometimes taken as the definition of "strong typing". Q: How is this affected by subtyping? If E:T, then the value of E should be a subtype of T Subsumption: if v:S and S subtypes T, then v:T. ** types in Scala Reference: Odersky et al, The Scala Langauge Specification 1.0 Odersky et al., An Overview of the Scala Programming Language (TR IC/2004/64) Gosling, Joy, Steele: The Java Language Specification *** security and types in network programming Soundness matters in Java, see the book "Java Security", by McGraw and Felten ------------------------------------------ WHY TYPE SOUNDNESS MATTERS OR HOW THEORETICIAL WORK CAN MAKE YOU FAMOUS class T { var x: SecurityManager; ... } class U { var x: MyObject; ... } var t: T; var u: U; // ...somehow make u and t aliases... t.x = System.getSecurity(); var m: MyObject = u.x; ------------------------------------------ the miracle is how to do the comment, if it can be done, then can make m a pointer to the system's Security Mmanager object, with type MyObject. Then all hell breaks loose *** subtyping **** definitions ------------------------------------------ SUBTYPING Def: S is a subtype of T, written S <: T, iff Properties of subtypes: ------------------------------------------ ... objects of type S are allowed to be used in any context that objects of type T are expected Q: What does this mean? ... - allow subtype objects to be used in place of supertype objects, i.e., S instead of T objects (subsumption) - no type errors if a subtype object is used as a supertype so can send messages if o: T, and if T has a method m(x:U):V then S must have a method m, that can be passed a U object, and will return an object that can be regarded as a V object so can S's m method take a subtype of U? A supertype? can it return a subtype? a supertype? - signature of subtype has to extend supertype's signature static types are an upper bound on the run-time type (i.e., any subtype is permitted) **** behavioral subtyping ------------------------------------------ AN ASIDE: BEHAVIORAL SUBTYPES Def: S is a behavioral subtype of T iff Properties of behavioral subtypes: ------------------------------------------ ... S is a subtype of T and whenever objects of type S are used in place of T objects, they obey the behavioral specification of T. ... no surprising behavior i.e., reasoning using static types (supertype abstraction) is valid **** inheritance and requirements on method typing ------------------------------------------ INHERITANCE AND METHOD TYPES class Int extends AnyVal; class Cell(xc: AnyVal) { private var xv = xc; def x: AnyVal = xv; def x_=(xc: AnyVal): Unit = { xv = xc; } } class PickyCell(xc: Int) extends Cell(xc) { override def x: = { ... } override def x_=(xc: ) = { ... } } val myC: Cell = new ...; val y = myC.x; myP.x = true; ------------------------------------------ Q: What can the return type of the PickyCell's x method be? AnyVal or Int, but not Any (covariant) Q: What can the argument type of PickyCell's x_= method be? Any or AnyVal, but not Int (contravariant) Q: Why? So that the same calls are legal, and still check Scala actually generates an overload if you change an argument type. What's the advantages or disadvantages of that? Q: What about exceptions? If they were recorded in method type (as in Java) could you allow more or less in the subclass? ------------------------------------------ STATIC OVERLOADING vs. DYNAMIC OVERRIDES (MESSAGE PASSING) class Foo { def f(x: Float): Int = 1; } class Bar extends Foo { def f(s: String): Int = 2; } // client code var x: Foo = new Foo(); x.f (3.14); x.f ("a string"); // static type error ------------------------------------------ the given methods are statically overloaded Explain how to subscript method names with their types to explain what happens with static overloading. One can also do the following, to make the types work ((Bar) x).f("a string"); Add a dynamically overridden method // Foo's def g (i: Int): int = 3; // Bar's def g (i: Int): int = 4; // client code: x.g(5) Now consider what happens if you add a new method to each, making g both statically overloaded, and some of the overloads dynamically dispatched. // Foo's def g (i: Int, p: Cell): int = 6; // Bar's def g (i: Int, p: PickyCell): int = 7; def g (i: Int, p: Cell): int = 8; *** names matter: classes and traits are types ------------------------------------------ CLASSES AND TYPES Every class and trait name ------------------------------------------ ... is a type name So this is "nominal" or "by name" typing Q: What does that mean for subtyping? it's also declared... ------------------------------------------ SUBCLASS ==> SUBTYPE class T {...} class S extends T {...} var x: T = new S(); ------------------------------------------ Q: Why doesn't Scala infer when a class implements a type or a subtype? so that types are behavioral, and so is subtyping *** casting ------------------------------------------ CHECKED CASTING E.g., var y : S = (S) randomExp; casting to a (reference) type generates ------------------------------------------ ... an exception if the object cast is neither null nor a subclass of that reference type (or does not have a class that implements the type) Note: no protected, private inheritance as in C++ classes can be declared abstract final classes can't be extended ** generics, variance annotations, and arrays source: Odersky et al's An Overview of the Scala Programming Language ------------------------------------------ GENERIC POLYMORPHISM class GStack[T] { var elems: List[T] = Nil; def push(e: T): Unit = { elems = e :: elems; } def pop: Unit = { elems = elems.tail; } def top: T = elems.head; } ------------------------------------------ Q: What's the advantage of this? *** An example ------------------------------------------ ARRAYS IN SCALA Suppose PickyCell <: Cell. Consider class Array[T] Is Array[PickyCell] <: Array[Cell]? ------------------------------------------ Look at the types of the fetch and store operations The fetch operation apply : Int -> Cell apply : Int -> PickyCell which is compatible with Array[PickyCell] <: Array[Cell] The store operation update : Int, Cell -> Unit update : Int, PickyCell -> Unit which is *not* compatible with Array[PickyCell] <: Array[Cell] Scala is more sensible than Java in this. In Java, PickyCell[] a subtype of Cell[]! So why isn't the type system unsound? Because there's a run-time check on the store operation! The actual store operation has the type __[__] = __ : Cell[], int, Object -> void throws ArrayStoreException __[__] = __ : PickyCell[], int, Object -> void throws ArrayStoreException if you try to store a Cell into a PickyCell[] array, then this exception happens at run-time This is clearly a mistake in the design, as type checking no longer statically guarantees against such simple type errors. So why was it done? Because they wanted subtyping for the read-only uses :-( *** variance annotations Q: How would you declare the relationships that might exist? ------------------------------------------ VARIANCE ANNOTATIONS trait VA[+D, -R, I] { val p: D; def save(r: R): Unit; var x: I; } D is covariant (read-only) R is contravariant (write-only) I is invariant (read-write) So VA[PickyCell, Cell, String] <: VA[Cell, PickyCell, String] E.g., case class Tuple2[+T1,+T2](_1:T1, _2:T2) trait Function1[-T0,+R] ------------------------------------------ Q: Why the annotation on the above examples? Q: When is a => b a subtype of c => d? Draw picture of this subtyping relationship. ------------------------------------------ CHECKING VARIANCE ANNOTATIONS Is this okay? trait MyFunction1[+T0,-R] extends Function1[T0, R]; Covariant (+) positions: + types of val members + return types of methods + type parameters that are covariant Contravariant (-) positions: - argument types of methods - upper bounds on type parameters - type parameters that are contravariant Invariant ( ) positions: . types of var members . type parameters that are invariant . types used in both co- and contravariant positions ------------------------------------------ Contravariance implies a flip of types: ------------------------------------------ FOR YOU TO DO Which of the declarations are consistent? abstract class VarAnnotaions[+D, -R, I] { var x: D; var y: R; var z: I; val a: D; val b: R; val c: I; def f(i: R): D; def g(j: D): R; def h(k: I): I; } ------------------------------------------ z, a, c, f, and h are okay ------------------------------------------ FOR YOU TO DO Which type parameters can be co- or contravariant? abstract class InferVariances[ A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P] { var x: A; val y: B; def f(z: C): D; def g: (E => F); def h(o: G): (H => I) => (J => K); def iterate(f: L => L, n: Int): L => L; def remember(e: => M): Unit; def hmm(h: => ((N => O) => P)): Unit; } ------------------------------------------ Correct list is: A,+B,-C,+D,-E,+F,-G,+H, -I,-J,+K, L,-M,-N,+O,-P Q: Can all of these be invariant? yes, that is a stronger restriction than allowed. ------------------------------------------ FOR YOU TO DO Is the following right? trait Super[+T] { def compare(x:Super[T]): Boolean; } trait Sub[+T] extends Super[T] { override def compare(x: Sub[T]): Boolean; } ------------------------------------------ method compare in Sub isn't an override and the variance annotations are wrong! Q: How could this be fixed? By using lower bounds... (skip details here, see below) *** parameter bounds **** upper bounds Q: GStack[T] works for all types T. What if you need more than the standard methods for the type parameter? ------------------------------------------ BOUNDED POLYMORPHISM trait Displayable { def display: Unit; } def displayAll[T <: Displayable] (elems: Seq[T]) = { for (val e <- elems) e.display } ------------------------------------------ Q: Why won't this type check without the bound on T? ------------------------------------------ F-BOUNDED POLYMORPHISM trait Ordered[T] { def < (x:T): Boolean; } def max[T <: Ordered[T]](x: T, y: T) = { if (x < y) y else x } ------------------------------------------ F-bounded means that the bound on a type T is a "function" of T like Ordered[T] **** lower bounds Example from An Overview of the Scala Programming Language by Odersky et al., p. 7 ------------------------------------------ COVARIANT TYPES AND IMMUTABLE OBJECTS trait GenList[+T] { def isEmpty: Boolean; def head: T; def tail: GenList[T]; def prepend(x: T): GenList[T]; // ok? } ------------------------------------------ Q: Is the definition of prepend legal? No, prepend is illegal, since T appears in a contravariant position Q: Can you create an example that shows why? Not really, as these lists are immutable (see why?), so we'd like GenList[S] <: GenList[T], if S <: T as shown in the following: trait Empty extends GenList[All] { /* ... */ } class Cons[+T](x: T, xs: GenList[T]) extends GenList[T] { /* ... */ } class E { def p: Unit; } class F extends E { def f: Unit; } val lf: GenList[F] = Cons(new F(), new Empty()); lf.head.p; // would be okay... val le: GenList[E] = Cons(new E(), lf); le.tail.head.p // would be okay... ------------------------------------------ FIX USING LOWER BOUNDS trait GenList[+T] { def isEmpty: Boolean; def head: T; def tail: GenList[T]; def prepend[S >: T](x: S): GenList[S]; } ------------------------------------------ Q: What does this say? Q: What happens if you use prepend with a subtype parameter? illegal Q: With a supertype parameter? legal Q: How can these be used to solve the problem with Super and Sub above? trait Super[+T] { def compare[U >: T](x:Super[U]): Boolean; } trait Sub[+T] extends Super[T] { override def compare[U >: T](x:Super[U]): Boolean; } ** type members and virtual types *** advantages ------------------------------------------ ADVANTAGES OF TYPE MEMBERS Dependent products: abstract class Cell { type T; var value: T; } class IntCell extends Cell { type T = Int; var value = 0; } Family polymorphism: - type member that vary together, covariantly E.g., parsers, FSEntityProblem, ... ------------------------------------------ Q: How is Cell like a generic type? abstract class GCell[T] { var value: T } Q: Why is family polymorphism useful? Know that relationships among types are preserved by overrides. *** problems ------------------------------------------ PROBLEMS WITH OVERRIDING TYPE MEMBERS abstract class Cell { type T; var value: T; } class IntCell extends Cell { type T = Int; var value = 0; } // Is this okay? class StringCell extends IntCell { override type T = String; } ------------------------------------------ Q: Scala doesn't allow StringCell, why? ------------------------------------------ PROBLEMS WITH TYPES FOUND THROUGH VARS object TypeInVarProblems with Application { trait TaggedValue { type T <: AnyRef; val value: T; } class Extractor { var v: TaggedValue = null; def extract: v.T = v.value; } val e = new Extractor(); e.v = new TaggedValue { type T = File; val value = new File("fn"); } val f: e.v.T = new File("fn2"); e.v = new TaggedValue { type T = String; val value = "hmmm"; } Console.println(f concat (e.extract)); } ------------------------------------------ Q: What's wrong with this? *** solution in Scala ref: Odersky et al., The Scala Langauge Specification, chapter 3 (draft of Oct 15, 2005) Q: What approaches are there to preventing these problems? don't let types and values mix (usual, e.g. Java) make sure types can't change in ways that cause such problems (Scala) check types at runtime (Lisp, Scheme, Smalltalk) ------------------------------------------ HOW TO AVOID PROBLEMS WITH TYPE MEMBERS 1. Don't allow overriding 2. Require type names to be ------------------------------------------ ... type definitions ... stable ------------------------------------------ STABLE IDENTIFIERS FOR TYPES object StableIds { abstract class B { val y = 3; object BO { type Smell = U; type Odor = V; } type U; type V = Float; class B2 { object o; } } class C extends B { type U = Double; object O { type OT = Int; val z : this.OT = 1; var x : C.this.U = 3.14; var w : C.super.V = 2.73f; var hmm : C.super.BO.Odor = 1.0f; } val q: O.OT = 7; val r: O.type#OT = 14; val s: C.super[B].BO.type#Odor = 5.41f; val t: super.B2 = new super.B2(); val u: t.o.type = t.o; } } ------------------------------------------ Q: What can't you use as a type? B.U, B.BO.Smell - because they are not known (can vary) the types of values, like z B2.o.type, since B2 isn't a value, it doesn't really have an o object in it yet (only instances do). ------------------------------------------ STABLE IDS AND PATHS Def: a *path* is one of: - \epsilon, the empty path - C.this, where C names a class (or trait) - this, which means C.this, where C is the enclosing class - C.super.x, where C names a class, and x is a stable member of its superclass - C.super[M].x, where M is a mixin of class C, and x is stable - super.x, which means C.super.x, where C is the enclosing class - p.x, where p is a path, and x a stable member of p Def: a *stable member* is either: - a package, - a val definition, - a class definition, - a type definition, or - an object definition Def: a *stable identifier* is a path, that ends in an identifier ------------------------------------------ Syntax: StableID ::= Id | Path . Id | super . Id | super '[' Id ']' . Id | Id . super . Id | Id . super '[' Id ']' . Id Path ::= | StableId | this | Id . this ------------------------------------------ TYPES FORMED FROM PATHS AND STABLEIDS Singleton types: syntax: p.type, where p is a path, p denotes a value, p conforms to AnyRef meaning: the set {p, null} examples: O.type t.o.type C.super[B].BO.type Type projections: syntax: T # x, where T is either a: - singleton type - a non-abstract class - a Java class and x is a type member of T meaning: the set of types defined by x in T examples: O.type#OT C.super[B].BO.type#Odor Type designators (names): syntax: sid, where sid is a stable identifier meaning: the type of values named desugaring: t ==> C.this.type#t, if t bound in C t ==> \epsilon.type#t, otherwise p.t ==> p.type#t Examples Float ==> scala.type#Float U ==> C.this.type#U this.OT ==> this.type#OT ------------------------------------------ ** typing mixin composition *** definitions ------------------------------------------ TEMPLATES Def: a *template* is what follows "extends" in a class or object definition. Def: A template, of form sc with mc1 with ... with mcn { ... } has *superclass* sc, *mixin classes* mc1, ..., and mcn Def: The *parent classes* of a template are the superclass and the mixin classes. ------------------------------------------ *** subtyping of mixins ------------------------------------------ SUBTYPING Subyping is declared. Given: class C sc with mc1 ... with mcn { ... } this produces: C <: sc C <: mc1 ... C <: mcn ------------------------------------------ *** defaults for parent classes ------------------------------------------ DEFAULT PARENT CLASSES Default superclass extends AnyRef Default mixin: with ScalaObject ------------------------------------------ *** problems from linearization ------------------------------------------ TYPING RESTRICTIONS ON TEMPLATES sc with mc1 with ... with mcn { ... } "The superclass of a template", sc, "must be a subtype of the superclass of each mixin class", mc1, ..., mcn. Why? trait Mixin1Super { def x = true } class Super { def x = 541 } trait Mixin1 extends Mixin1Super; class Sub extends Super with Mixin1; !(new Sub().x) // ok? ------------------------------------------ The linearization goes: Mixin1Super | Super | Mixin1 | Sub This linearization is fine if Super <: Mixin1Super, as in: trait Mixin1Super { def x = true } class Super extends Mixin1Super { override def x = false } trait Mixin1 extends Mixin1Super; class Sub extends Super with Mixin1; !(new Sub().x) *** self types From an Overview of the Scala Programming Language by Odersky et al. ------------------------------------------ SELF TYPES In a class or object declaration, can declare the type of "this": class C : s extends sc with mc1 with ... with mcn { ... } Def: s is the *self type* of C. Default: C Restrictions: 1. The self type s must satisfy: s <: sc's self type s <: mc1's self type ... s <: mcn's self type 2. Each expression of form new D(...) must be such that D's self type is a supertype of (or equal to) D. ------------------------------------------ Q: Why these restrictions? What could go wrong? ------------------------------------------ EXAMPLE WITH SELF TYPES trait SubjectObserver { type S <: Subject; type O <: Observer; abstract class Subject: S { private var observers: List[O] = List(); def subscribe(obs: O) = observers = obs :: observers; def publish = for (val obs <- observers) obs.notify(this); } trait Observer { def notify(sub: S): unit; } } ------------------------------------------ Have to prevent Subject and Observer from directly refering to each other, to allow covariant extension of both. ------------------------------------------ USING SubjectObserver object SensorReader extends SubjectObserver { type S = Sensor; type O = Display; abstract class Sensor extends Subject { val label: String; var value: double = 0.0; def changeValue(v: double) = { value = v; publish; } } abstract class Display extends Observer { def println(s: String) = ... def notify(sub: Sensor) = println(sub.label + " has value " + sub.value); } } ------------------------------------------