Carlo Scarioni

Explore a simple example of writing a Pulumi provider for Airbyte. Instead of using the official Terraform provider, implement a Pulumi provider in Python.

Scala offers a functionality known as Structural Types which allows to set a behaviour very similar to what dynamic languages allow to do when they support Duck Typing (http://en.wikipedia.org/wiki/Duck_typing) The main difference is that it is a type safe, static typed implementation checked up at compile time. This means that you can create a function (or method) that receives an expected duck. But at compile time it would be checked that anything that is passed can actually quack like a Duck. Here is an example. Let’s say we want to create a function that expects anything that can quack like a duck. This is how we would do it in Scala with Structural Typing: def quacker(duck: {def quack(value: String): String}) { println (duck.quack("Quack")) } You can see that in the definition of the function we are not expecting a particular class or type. We are specifying an Structural Type which in this case means that we are expecting any type that has a method with the signature quack(string: String): String. So all the following examples will work with that function: object BigDuck { def quack(value: String) = { value.toUpperCase } } object SmallDuck { def quack(value: String) = { value.toLowerCase } } object IamNotReallyADuck { def quack(value: String) = { "prrrrrp" } } quacker(BigDuck) quacker(SmallDuck) quacker(IamNotReallyADuck) You can see that there is no interface or anything being implemented by any of the three objects we have defined. They simply have to define the method quack in order to work in our function. If you run the code above you get the output: QUACK quack prrrrrp If on the other hand you try to create an object without a quack method and try to call the function with that object you would get a compile error. For example trying to do this: object NoQuaker { } quacker(NoQuaker) You would get the error: error: type mismatch; found : this.NoQuaker.type required: AnyRef{def quack(value: String): String} quacker(NoQuaker) Also, you don’t even need to create a new type or class. You could use AnyRef to create an object with the quack method. Like this: val x = new AnyRef { def quack(value: String) = { "No type needed "+ value } } and you can use that object to call the function: quacker(x) You can also specify in the function that expects the structural type, the the parameter object must respond to more than one method. Like this: def quacker(duck: {def quack(value: String): String; def walk(): String}) { println (duck.quack("Quack")) } There you are saying that any object you pass to the function needs to respond to both methods quack and walk. This is also checked at compile time. Under the covers the use of Structural Types in this way will be handled by reflection. This means that it is a more expensive operation than the standard method call. So use only when it actually makes sense to use it.

The Map Reduce Framework works in two main phases to process the data, which are the "map" phase and the "reduce" phase.