Testing Clojure

07/03/2010

My latest pet project has been a little Clojure program that screen scrapes the TFL Oyster web site and calculates my (or yours) daily Oyster expenses. The project is hosted here: http://github.com/erlingwl/Oyster.

It took me a while to get comfortable with Clojure and to actually become able to write some tests. I had very much the same experience as my colleague Sam Newman writes about.

However, once I actually started doing proper TDD, I found my program evolved quite naturally. The problem might be that some unit tests are testing single functions, whilst others are testing more aggregate functions. My initial thought on this is that probably some of my unit tests are closer to what you might call acceptance tests. However, I am not sure how well this approach would scale, and as Sam mentions, one might have to stub out the sub functions in order to only test the specific function..

If you take a look at the source code, you can see I used two different ways of testing my app traditional unit testing using clojure.contrib.test-is and cuke4duke which is cucumber for various languages on the JVM including Clojure. As there is not too many examples of how to test Clojure out there, hopefully someone might find it useful to take a closer look at what I have done so far. The cuke4duke scenarios are not very impressive in their current state though, as they were amongst the first things I wrote. A reasonable approach forward would be to convert some of the acceptance test like unit tests into scenarios.

Anyway, playing with Clojure has been fun, and it has given me valuable insight regarding the future of languages on the JVM, expect a blog post on this topic soon!

A Kanban Brown Bag Recipe

03/12/2009

Kanban appeals to me, and I have had the privilege to hold a brown bag on it two times in the last weeks. The feedback and especially the discussions it started exceeded my expectations in a very positive way. Therefore, I thought it might be worth sharing my recipe for a Kanban brown bag.


The materials you need:

  • Henrik Kniberg’s Kanban kick-start example. (Thank you very much for sharing this Henrik! It would have taken ages to prepare the brown bag without it)
  • A keyword cheat sheet
  • 20 coins
  • 5 stopwatches (or mobiles)
  • 8 participants + yourself

Step 1: Presentation
Connect your computer to the projector and open Henrik’s kick-start example. Scribble down the following keywords on a flip chart as you explain how they are illustrated with Henrik’s example:

  • Visibility – The Kanban board. Visualize bottlenecks
  • Limit Work In Progress (WIP) – Avoid task-switching. Focus on throughput rather than over-utilizing individuals
  • Pull – You are not allowed to push work down the workflow, you pull work when you are ready and are below WIP limit
  • Flow – A steady flow (all the way from analysis to production), not iterations
  • Cadence – You measure and predict, instead of plan and promise

Assuming you have a basic understanding of Kanban and agree with my selection of keywords, your audience should now have learned the basic concepts of Kanban. To make them fully grasp the value of limiting work in progress the next step is to play a simple game:

Step 2: Play a game
I learned this game at XTC (once again thanks to Henrik Kniberg who facilitated the game at XTC).

Basically you start with four players, and four “managers”, standing behind one player each with their stopwatches ready. The first player gets 20 coins. His job is to flip all 20 coins, then pass them on to the next player, which will then flip all 20, then pass them on etc.. Each manager measures the time, from when his player starts flipping the first coin, until he has flipped all 20. You as a facilitator, should measure the time from the first person flips the first coin, until the last person flips the last coin. The initial scoreboard would look like this:

Batch size Player 1 Player 2 Player 3 Player 4 Total time
20
5
1

The next round, the first person flips 5 coins, passes them on, flips the next 5 etc. Each manager measures the time from his player flips his first coin, until he passes the last coin on. The next person starts flipping the 5 coins as soon as he receives them, passes them on, then starts on the next 5 when available… You as a facilitator, measures the total time. The last round, the first player flips one single coin, passes it on, then flips the next..

If you don’t want to know the typical outcome of the game, stop reading now!

The outcome of the game will most likely look something like this:

Batch size Player 1 Player 2 Player 3 Player 4 Total time
20 9.8 11 13 14 56.4
5 16.7 17 21.5 19 32.6
1 14.9 21 23.1 22.3 26.3

As you can see, the individual times is higher for the smaller batch sizes. However, the total time is much better for the smaller batch sizes.

Batch size Methodology
20 “Waterfall”
5 “Scrum”
1 “Kanban”

I have indicated next to each total time which process the different batch sizes could relate to. Hence, tell your audience that if they want to improve their overall throughput, they should limit their WIP and give Kanban a try 🙂

Thanks again to everyone in the community who have shared their material and knowledge.

My polyglot exploration

10/11/2009

Having used Java as my main language lately, it was about time to play with some new languages. As there is a whole stack of languages that runs on the JVM now, it is hard to decide which language to play with – so I ended up playing with them all, or at least the following; JRuby, Jyhton, Groovy, Scala and Clojure.

Although they all run on the JVM, I wanted to find out how easy it actually is to combine them. My method was almost like this:

  • Install the programming language (thank you macports)
  • Find and install the correct plugin for Eclipse
  • Try execute some Java Library / code from the language
  • Try to execute something written in the language from Java
  • On some occasions I also tried to execute some of the other languages from the current language I was playing with.


    Disclaimer: I just want to share what I have learnt so far. There might be way better ways to do these things.

    Groovy
    To install Groovy I just did sudo port install groovy, whatever platform you are on, it shouldn’t be too hard.

    I also installed the Groovy Eclipse Plugin and created a new Groovy Project.

    Here is the Groovy code I wrote for calling Java

    package groovyerlingwl;
    
    import java.util.regex.Pattern
    
    class CallingJava2 {
    	
    	def pt = Pattern.compile("^foo")
    	def m = pt.matcher("foo")
    }
    
    def callingJava = new CallingJava2()
    println  "" + callingJava.m.find()
    

    To call some of my Groovy code from Java, I created a simple class:

    package groovyerlingwl;
    
    class GroovyPrinter {
    	
    	def printsomething() {
    		println "hello from Groovy"
    	}
    
    }
    

    ..and exported it as a .jar file. Interestingly enough, the .jar file now contained a groovyerlingwl/GroovyPrinter.class. To run this from Java, I had to add groovy-1.6.5.jar and asm-2.2.3.jar to the classpath. I found mine in /opt/local/share/java/groovy/lib. And all the code you need is:

    import groovyerlingwl.*;
    
    public class CallingGroovy {
    
    	public static void main(String[] args) {
    		GroovyPrinter gp = new GroovyPrinter();
    		gp.printsomething();
    	}
    	
    }
    

    Scala
    Very much like Groovy, Scala integrates simply with Java. After installing Scala, the Scala Eclipse Plugin and created a new Scala Project; I used the following to call Java:

    package erlingwlscala;
    
    import java.util.regex.Pattern
    
    object Test {
    
      def main(args: Array[String]) { 
        val pt = Pattern.compile("^foo")
        val m = pt.matcher("foo")
        println(m.find)
      }
      
    }
    

    And then I made another simple class to call from Java:

    package erlingwlscala
    
    class TestPrinter {
    
      def print_something(){
         println("Hello, from scala") 
      }
      
    }
    

    Again, I exported this to a .jar file. This also resulted in the .jar file containing a erlingwlscala/TestPrinter.class.

    To run the class from Java, I added scala-library.jar to the classpath and ran the following Java code

    import erlingwlscala.TestPrinter;
    
    public class CallScala {
    	
    	public static void main(String[] args) {
    		TestPrinter testPrinter = new TestPrinter();
    		testPrinter.print_something();
    	}
    
    }
    

    JRuby
    Again, I installed JRuby and the matching Eclipse Plugin. I created a new Ruby Project. However, make sure you choose the JRuby interpreter when running the following code:

    require 'java'
    
    include_class 'java.util.regex.Pattern'
    
    class CallingJava
      
      pt = Pattern.compile("^foo")
      m = pt.matcher("foo")
      puts m.find
      
    end
    

    Running JRuby from Java was not as easy as Groovy or Scala. However, using the JSR-223 it seems that you can run almost any scripting language from Java. So using the approach described in the article you can do the following in Java:

    import javax.script.*;
    
    public class CallJRuby {
    
    	public static void main(String[] args) throws Exception {
    		ScriptEngineManager factory = new ScriptEngineManager();
    
    		ScriptEngine engine = factory.getEngineByName("jruby");
    
    		try {
    			engine.eval("puts('Hello from JRuby')");
    		} catch (ScriptException exception) {
    			exception.printStackTrace();
    		} 
    	}
    
    }
    

    NB! Remember to download The JSR engines and include the jruby-engine.jar on the classpath. You also need to include jruby.jar on the classpath.

    Clojure

    Again, installing the suiting Eclipse Plugin. And creating a new Clojure plugin. I used the example from Clojure.org and borrowed the following code:

    (doto (new java.util.HashMap) (.put "a" 1) (.put "b" 2))
    

    To call Clojure from Java, I found this article.. First make a simple clojure program

    ; printer.clj
    (ns printer)
    
    (defn print-string [arg]
    	(println arg))
    

    Add this to your resources folder in your Java project, include clojure.jar and clojure-contrib.jar on the classpath, and run the following code:

    package erlingwl;
    
    import clojure.lang.RT;
    
    public class CallingClojure {
    	
    	public static void main(String[] args) {
    		try {
    			RT.loadResourceScript("printer.clj");
    			RT.var("printer", "print-string").invoke("hello world");
    		} catch (Exception e) {
    			e.printStackTrace();
    		}
    	}
    }
    

    Jython

    Once again, I installed Jython and downloaded the respective Eclipse Plugin. To call Java I created a new Jython project and used the following code (got it from wiki.python.org):

    from java.util import *
    r = Random()
    print r.nextInt()
    

    wiki.python.org also describes how you can use JSR-223 to run Jyhton/Python from Java. I tried the following code:

    import javax.script.ScriptEngine;
    import javax.script.ScriptEngineManager;
    import javax.script.ScriptException;
    
    public class CallJython {
    
    	public static void main(String[] args) {
    		ScriptEngine engine = new ScriptEngineManager().getEngineByName("python");
    		try {
    			engine.eval("print 'hey from python'");
    		} catch (ScriptException e) {
    			e.printStackTrace();
    		}
    	}
    }
    

    To get this to work, add jython.jar and the jython-engine.jar from The JSR engines to the classpath.

    Bonus
    I mentioned I tried to combine some of the other languages too.

    Clojure calling scala:

    (println (doto (new erlingwlscala.TestPrinter) (.print_something)))

    To do this, I added my scala application .jar to the classpath (see above where I call Scala from Java), and also I had to add the scala-library.jar.

    JRuby calling clojure:
    Here I basically followed the same approach approach as when I called Clojure from Java (see above):

    require 'clojure.jar'
    
    include Java
    
    import 'clojure.lang.RT'
    
    class CallingClojure
      
      RT.loadResourceScript("resources/printer.clj")
      RT.var("printer", "print-string").invoke("hello world")
    
    end
    

    Calling Scala from JRuby:

    My worst hack so far, I assume there are more elegant ways to do this:

    include Java
    
    SCALA_HOME =  '/opt/local/share/scala/lib/'
    require SCALA_HOME + 'scala-library.jar'
    require "scalaexport1.jar"
    
    include_class 'erlingwlscala.TestPrinter'
    
    class CallingScala
      
      printer = TestPrinter.new
      printer.print_something
      
    end
    

    How to find a flat in London

    10/09/2009

    Not sure if it’s a god or bad sign that my blog is currently taking a non-technical direction. However, finally having moved into our new flat in London, I thought I should share some of my experiences – maybe someone will find this useful one day.

    Having decided to move to London (from Norway), we started looking at websites such as Loot, Gumtree and TimesOnline. However, it seemed to us that most of the flats were available immediately – hence we decided to start looking more seriously when we first arrived.

    So, we booked a cheap hotel for the first week. Took some walks around the area we wanted to live in, and searched as crazy on the websites. We soon realised we had to settle for a quite small flat (read: studio) if we wanted to live in central London. After a day or two, we found an ok flat, went for a viewing and decided we wanted it. We soon realised that bringing our dog turned out to be quite a challenge. The landlord was okay with the dog, however the leaseholder (the council) needs to grant permission. The real estate agents were really helpful, however, since we are both busy people and not staying home with the dog all day, they told us our chances were minimal.

    Having wasted a day or two hoping to get this flat, we realised we needed to widen our search (Tip #1: Don’t put all your eggs in one basket). We had scanned every ad on the internet, so we decided to walk around a bit. We phoned every number on the “to let” signs we could see. By doing this we finally found a flat that suited us.

    Ironically this flat was priced a bit higher than what we could afford, so we had probably filtered it out when we searched on the internet. However, the real estate agent immediately offered it to a price more suitable to our budget (Tip #2: Prices are negotiable). But of course, he needed to verify that the dog was okay etc.

    Having learned from our mistakes, and being aware of our dog might be a challenge. We started searching further north. This would force the both of us to take the tube or such, meaning we had to calculate this into our budget. We found some dog friendly flats – but it wasn’t exactly in the area we had hoped to live. Hence, we were very pleased when we got a call from the second real estate agent, telling us that everything was okay with the dog. Still we needed to fill in a form for credit checking..

    By this time we were tired of living at a tiny hotel room – we booked a new hotel (got a bargain at hotels.com) for another week. This hotel was a lot closer to our area of interest and it was much nicer (Tip #3, find a hotel nearby your area of interest, allow yourselves some luxury). It meant we had to transport our 100kgs of luggage once again though.

    Due to the Bank holiday, a busy real estate agent who promised a lot more than he could keep and some unskilled credit checking company (who struggled to make a simple phone call for two days) – our credit check wasn’t done until late friday (the second week). By then our real estate agent had turned off his phone for the weekend..

    Hoping to move in early in the week, we extended our stay at the hotel for two nights. Booking directly at the hotel was expensive, but it was at least only for two days. Or so we thought.. It turned out we had to stay yet another night at a hotel – waiting for some furniture to arrive. By then the hotel rates on our current hotel had exploded. We got permission to place our luggage at the flat however, before we booked a cheap hotel for one last night (Tip #4, if you find a nice hotel, at a decent rate, book it for 2,5 times as long as you think you will need!).

    We have spent our first night in the flat now. Sleeping at the sofa bed mattress, while waiting for the bed to arrive. There are still some things to sort out, such as opening an UK bank account, choosing an electricity provider, get an internet connection up and running etc.  however, having a home in London now is fantastic. I look forward to enjoy each new day! Might be continued..

    Starting at ThoughtWorks

    09/09/2009

    As I wrote earlier, we have moved to London – leaving my wonderful colleagues at Miles. My girlfriend will be studying at UCL, and I’m extremely happy to be starting at ThoughtWorks in October.

    ThoughtWorks’ recruiting process, with the coding task and a whole day of tests and interviews, was challenging – but at the same time it made working at ThoughtWorks even more appealing. I got to meet quite a few of my new colleagues and they all confirmed my impression of ThoughtWorks as a fantastic place to work. For me this is if a dream has come true. To be continued..

    Ps. If you are looking for any tips on getting hired at ThoughtWorks I would recommend you to find someone at ThoughtWorks you either know or somehow can relate to – they might help you through the first stage of getting your CV accepted (Thanks for helping me!). After that, just be yourself and do your best =)

    Adopting Agile – Challenges with the bottom-up approach

    06/07/2009

    Even though someone has probably written about this before, this is something I have experienced to a certain degree and feel I would like to share  with other Agile evangelists.

    I’ve read a lot about Agile being adopted with the bottom-up approach. Developers wants to do their jobs better, faster and deliver more business value more frequently. They start exploring Agile approaches such as continuous integration or test-driven development. Sooner or later a couple of evangelists have convinced their team-leader or project manager to try out agile on their next project. They find themselves creating a sprint backlog, having daily stand-ups and maybe even retrospectives.

    But hey – what about the demonstration meetings and the product owner? If the business people that are supposed to prioritize the product backlog are not familiar with agile, or are organized in a way that makes it hard for them to agree on a prioritization of the user stories in the product backlog, the project managers or the architects have to take on the customer role. This is a compromise that may very well work out okay. But it is hard to know if it the developer team really is delivering the optimal pieces of business value at the end of each iteration. Meanwhile the organization looses some of the benefits of their teams doing agile – for instance the possibility to turn around or change course quickly.

    My point is  that using the bottom-up approach for adopting agile, will force you to make compromises that will affect the benefits of agile. Sooner or later the adoption may spread further up the organization – or if you’re really lucky someone might start a top-down approach too and meet you half-way (not that I know how that would work out =)

    Searching for the truth of group dynamics

    21/06/2009

    Every now and then you discover or learn something that you believe is a sound statement, fact or pattern. Something which is true and will stay true even after a long time. Regarding group dynamics, I believe the following are such universal facts:

    • Endre Sjøvold, a professor at NTNU I have deep respect for, states at his website (freely translated from Norwegian): “..when you believe that learning a specific technique is enough to ensure success, is when it actually fails.  It is always so that the balance between technical skills and a genuine willingness to work together is what ensures success.” He says this in the context of looking at the history of hyped methodologies from team building to SCRUM. If you’re an innovator like me, who loves to embrace new ideas, try to keep Endre Sjøvold’s statement in mind whenever you get confused, caught up in too much details or feeling your intuition tries to tell you that something is wrong.
    • Linda Rising and Mary Lynn Manns’ Fearless Change: Patterns for Introducing New Ideas. If you haven’t read this book or been to any of Linda’s tutorials yet, you should really try it! People are not rationale, you cannot convince them with traditional argumentation, instead you should find their guru and feed him with chocolate chip cookies (amongst other patterns).

    Finally I believe Joseph Pelrine is really onto something with his heat model. His talk at QCon London was great, and I’m really looking forward to his book. Meanwhile I’ll continue my search..

    London calling

    17/06/2009

    It’s official, we’re moving to London!

    My clever girlfriend Anne-Linn, will be studying Speech Therapy at UCL from September. Hence we’re moving to London in late August. I’m excited!

    Hopefully I’ll find a thrilling place to work and meet lots of interesting people. If you need a passionate software developer who cares about people, please contact me on erlingwl(at)gmail.com. I’m going to miss my fantastic colleagues at Miles though, you guys have been extremely good to me!

    Will I be the Innovator forever?

    06/06/2009

    I love to embrace new ideas, some would say a bit too much, others would say I challenge them and provide valuable input. I enjoy trying out the latest technologies as well as methodologies. Thanks to Linda Rising I also know that people are not rational – hence if everyone doesn’t buy in on my suggestions all the time, I know I could always try another approach (e.g. feed people chocolate chip cookies..).

    However, I learned at the university (especially from Endre Sjøvold)  that members of a mature team does not step into static roles. They adapt to their environment, and acts out the roles that are necessary at specific points in time. So if I find myself taking on the role of being an Innovator for most of the time, does that mean I’m part of an immature team? If so, am I a mature as a person, and as I spot the lack of an Innovator I take on that role? Or do I just do it because I have a preference for taking on the role of being an Innovator? And could that stop me from being a sceptic (or in Linda’s terms, I think; a Laggard) when it is needed of me?

    Working as a software consultant in the financial industry, I’ve discovered most people doesn’t tend to take on the roles of being Innovators. But what would happen if I were put in a team where everybody loved being Innovators? Would I put on a more conservative hat when needed? I believe it would be possible for me to do that, but I’m of course afraid that my passion for learning new things would challenge me on that. I think it would be especially hard for me to take on a long term role of being more conservative. Hopefully my other team members would have relieved me from my duties from time to time, since they would be just as mature as me (probably even more :).