Mr. Legacy

At my current (at the time of writing) work place I am often referred to as Mr. Legacy. This is a title that I’m honoured to have earned. But how did that come about?

I was hired to do some fire fighting. My boss knew me from a previous assignment and it was initially meant to be a short assignment, but I liked it there and they seemed to like me, and one thing led to another, so I stayed on board for – so far – more than 7 years.

For some reason I have a knack for messing around (in the good way) with old crap, that’s just saying it as it is. I’m very good at surgical incisions, “spot-improving” the code and adding features without otherwise ruining the system. And I’m not just good at it. I actually like it!

But there was a time when even I was bested. I’m not proud of it, for several reasons, which all will come clear.

It all started in 2008 when I got a call from my job-broker: Would I like a small, cosy 120 hour project, implementing a few features in an existing project, written in Borland C++? Of course I would since I hadn’t been working for a couple of months.

The application was a automatic dutyroster planner managing dutyrosters for employees who work different shifts and takes into account various union rules, employees wishes, overtime etc. The built-in autoplanning feature worked surprisingly well.

At the time they had made some promises to a customer about the application. It should be ablt to not only plan the shifts but also record which hours the employees were actually there.

My job in that respect was to implement an export of those hours to a third party accounting system. A simple job, you would think, but I was in for a surprise.

My first action was to take a look at the database. The data obviously had to be there somewhere to make en export happen. There where some tables that had the right smell, but did not fit the description exactly. After a few hours I had to give in – I could not figure out how this was supposed to be done. And at that time I was pretty certain that it would take more than the 120 hours to implement, because the application did not seem to even have the features for recording the hours.

I had a meeting with the boss. Told him my view of the lay of the land. At the same time he was both shocked and not surprised at the same time.

It was not thought through at the moment, but I suggested we made some kind of partnership deal because I really did think there was something valuable in that product. Great ideas, great functionality and existing customers. We came to an agreement, details of which are not relevant here.

Then started to ordeal. Another requirement from the customer was to enable multi-user support. Had I known about that requirement I might definitely not have entered into the agreement.

The application used a proprietary database – EasyTable (nothing to do with the name of the application) – which is really a quite decent database. And it sure was easy to use.

But in the application there were no sight of a data access layer. There was littered SQL all over it. Concatenated SQL. Not parameter bindings. At all. Table and field names was repeated all over, too.

To enable multi-user support we had to use another database, wherefor we needed to become more or less database agnostic. In other words: A DAL was needed. A database independent one. A task that in itself is not that difficult.

I wrote some basic Entity and EntityList classes and quickly figured out how to implement some C++ template classes to help me out. Boring job, but it had to be done.

The really hard part was to change existing embedded SQL-statements and field names as magig strings to using Entities and EntityLists.

The application used a DataModule form on which TQuery-thingies (Delphi did the same, if I remember correctly) was put more or less resembling the actual tables in the database. Mostly less. These tables were generally abused all over the application to hold some random SQL and return the corresponding recordset or to update various data in the database.

The main form in the application had about 10.000 lines of code behind. A staggering number of lines, but if it were well-written it would have been manageble. It was not. Global variables were as abundant as parenthesis in LISP. And they were referenced from everywhere, even other forms.

The application was a night mare, it was surprising it worked as well, as it did.

Eventually I had to throw in the towel. After I had sunk 100s of hours into the application I gave up. I was cleaned up exceptionally well at the time, but admittedly I ran out of money not to earn while working on it.

Lessons learned

First I’ll take a look at what went wrong for the application in the first place. What thoughts went into choosing C++ Builder for the task? At the time, in 2008, it would have been considered insane, but we have to look at it in the context of 2003 where they started the application. What where the options back then?

.NET 1.1 came out officially 9. April 2003 (https://en.wikipedia.org/wiki/.NET_Framework_version_history) and really was not useful for much before then. I do not remember the state of the Winforms designer, but I guess it was just as good as the VB6 designer, which was horrible, but was used to create loads of applications.

The other option was Delphi, also from Borland. I guess the form designers where about the same level and I was pretty fond of the designers in Delphi 1 to 6.

The people making the choice was not very experienced in the business. They were just out of school and there chose what they had used in class: Borland C++ Builder, which anyone who have used it will readily admit is not a good product. But still, I understand why they made the choice.

Whatever tool they had chosen it would not have saved them from the flood of mistakes they continued to make. Again, it was probably understandable. The application started as a proof of concept and continued to grow and had a demand so they never could get around to refactor it.

But did I learn something from this quest? I have to say yes. It was hard earned, so having nothing learned would be sad.

In hindsight I should, of course, have left the assignment after the first few hours but what could have warned me of that back then?

Firstly I probably should have spend a considerable amount of hours trying to understand the innards of the application before entering any agreement about it. That I did not do. Today I would – in the same situation – simply state that the wanted functionality could not be implemented with the current state of the application.

I should, also, have talked to some customers – or at least the boss – about expected and known feature requests, so I could have judged whether they were feasible.

Learning F#

I have started on a new quest. I want to learn F#. And I want to as good in F# as I am in C#. Well, you might say, how hard can it be? It’s just another language. Just my initial thought, sort of. There is one huge difference compared to C#: F# is (firstly) a functional programming language, whereas C# is an object oriented programming language. And to me that is a considerable obstacle. Let me be honest here: the one course in my computer science study that concentrated on a functional language (Miranda) took me three tries to pass. Not something I’m particular proud of, but it sure is an indicator how hard it was for me – I am – honestly – not that stupid 😉

There are several issues with learning a new language.

1. The libraries and frameworks available

Since F# runs on the .NET (and Mono) platforms this is an issue that can mostly be overlooked. F# can call everything .NETish, and F# is a hybrid language, so various object-oriented frameworks (in C#) can be used, too.

2. The eco-system, IDE, tools etc.

Some people think that an IDE is not a good idea. I’ll leave them to struggle with that. I like when my computer is helpful, especially when learning something new. F# is supported in Visual Studio, but the support is light years behind that of C#. There is not, for instance, something as simple as a rename-refactoring support.

The tools available are vastly different, as well. I’m used to Resharper and uses its test runner when running unit tests. Since Resharper don’t have the faintest idea what F# is this limits the usefulness of the test runner quite a bit. It can be used, though.

The F# compiler is wildly more intelligent and smart than the C# compiler. It almost does, out of the box, what Roslyn is supposed to give to C#. The editor in Visual Studio uses the F# compiler to continuously compile the code and underlines (with red squiggly lines) problematic code, without you hitting compile at all. Something like what Resharper does when “Whole solution analysis” is enabled. Quite neat.

3. The paradigm and syntax

This is where I struggle the most. Functional programming is a vastly different paradigm than object oriented. And even though Microsoft has sneaked functional constructs into C# over the years, I still struggle with going all the way.

And in addition to have to think in a wholly different way about how to solve a problem I also have to get used to a wholly new syntax – both reading it and writing it. Here I am truly happy about using a fairly smart IDE, because even having it, I can sometimes ponder for minutes over some error, before getting it.

4. Structuring the code

After using mantras like “one class, one responsiblity“, one class per file, SOLID, Design Patterns, libraries, layers and loads of other principles to keep me out of harm’s way I’m now looking at a totally blank slate. Since functional programming still not has the benefit of being main stream (getting there, but not yet) the collective wisdom of the very good functional programmers still haven’t been distilled into sound advice, so each and everyone is more or less on their own. This fact is intensified by something else I have noticed when meeting other functional programmers:

5. The milieu

The functional milieu seems to be – in some respects – a very mathematically oriented, elitist milieu, which – in its approach to programming – is very far from the many sound principles which seem to proliferate in the object oriented milieu. And this is strange. A lot of the things that good programmers do in object oriented languages, like using meaningful names, structuring the code in meaningful ways, functional programmers seem to regard with contempt. They seem to think along the lines of: Why write let CalculateTax income = ... when you can simply write let ct i = ....

This is, of course, overstated a bit, but the write once, read many mantra still have a long way to go in the functional milieu and the general elitist attitude will work against adopting functional languages, since the average programmer will simply look at it and say: It’s to hard. And to make a programming language main stream it has to be adopted by the average programmer.

How am I going about it?

I have been wanting to get into F# for a few years, now. It started to get serious, when I in 2013 attended the Skillsmatter Fast Track to F# with Tomas Petricek and Phil Trelford. I then went to New York to attend the F# Tutorials, which I also plan to attend in 2014.

Then I bought the book (along with others): Functional Programming with F#, which has exercises in it and seems to be a good book to teach me the building blocks of F#.

In Copenhagen a functional meetup has started, where we, amongst other stuff, has a monthly meetup where we in groups work through the book. In connection with that I am exposing myself by uploading my feeble attempts at solving the exercises to github.

I’m also trying to work at a more or less real world problem: url routing and html templates, where I try to focus on creating a nice API for the user of the library and on structuring the code in a nice, consistent and readable manner. Definitely a work in progress!

So all in all I’m attacking the problem from various angles and I feel I’m getting there – slowly, but surely!

I really want to be a skilled, versatile FSharp Dressed Man :D!

Specifying Business Rules by Example

One of the (many) benefits of using Specification by Example is that we get the business logic out in the open, being able to discuss it among developers, analysts, domain experts and users, instead of having it buried deep down in the code, wherefrom it can be rather hard to extract. And especially hard to extract non-manually.
 
But I think it is really hard and difficult to figure out from which perspective (some of) these business rules should be written. I mean, some business rules do belong in the abyss of the code, but other rules should most definitely be surfaced and be easy to extract as documentation.
 
To encourage some thoughts about this, I will describe my thoughts about a more general problem, which many systems will face. The problem, I think, is generic enough that the discussion it might spur will be useful in other situations.
 
We have some system (The System), which should be accessible by web (The Web Site) and mobile apps (The App). The System demands that Members do Create an Account. And for (more or less) obvious reasons The System has some requirements as to what makes up a Valid Username and a Valid Password.
 
I see the action of creating an account via The Website and The App as two different features (e.g. stories) of The System. So we would have two features for describing this, very much alike:
 
Feature: Create Account on The Web site
    As a potential Member of The System
    I want to Create an Account on The Web Site 
    so that I can login and use the features of The System.
Feature: Create Account via The App
    As a potential Member of The System 
    I want to Create an Account in The App 
    so that I can login and use the features of The System.

I believe I have managed to describe these two features in a way that open up for writing Scenarios suitable for automatic testing. A Scenario for the first Feature could be:

Scenario: Successfully Create Account on The Web Site
    Given a Valid Username that is not already in use and a Valid Password
    When The User asks The Web Site to Create the Account
    Then The Account is created by The System and the User is logged in as a Member of The System    

 

The Scenario for The App will be very much the same. And I figure that these Scenarios should be automatically tested through their corresponding UI.

 

My brain starts to heat up, though, when I think of the actual business requirements as to what makes up a Valid Username and a Valid Password. It should be obvious that these rules should be upheld no matter whether we create an account using The Web Site or The App. And since I will want the Create Account Scenarios above to be tested through UI, which is (very) slow, I don’t want to list several examples of Usernames and Passwords, which then all should be tested through the UI.

 

The App will (obviously?) use a Rest API for creating and accessing account information. The REST Api will use an Account Application Service, which also will be used by The Web Site’s server-side code (The Web Site could also use the REST Api AJAX-style, but for now we assume it doesn’t). Note, that this also means that we want to – somehow – test the REST Api, using the same business rules, which would result in one more Feature, describing that Scenario.

 

So what I’m actually aiming at here is that I want the Account Application Service to be Specified by Example, as well. And I want this because the rules that describe what constitutes a Valid Username and Valid Password are business rules, open for discussion. They should not be buried in the code. At the same time we don’t want to repeat the description of the business rules all over Features and Scenarios.

 

I cannot see a clear-cut way to this nicely and it may be that I’m using the wrong hammer (in this case SpecFlow, which may be hindering me thinking outside the famous box) for this or just my inexperience in Specification by Example shining through.

 

Since what I want to Specify is the Account Application Service I will call the users of the service for Consumers. What I have come up with is this:

 

Feature: Validate Username and Password
    As a potential Consumer of The System's Account Application Service 
    I want The Account Application Service to Validate my Username and Password according to the Business Rules.
    so that I can create an account with a Valid Username and Valid Password 

A Valid User Name:
  1. Cannot begin with white space.
  2. Cannot end with white space.
  3. Must start with a Unicode letter
  4. Must consist of letters, digits and spaces.
  5. Is case insensitive.
  6. Is at least 3 characters long.
  7. Is at most 30 characters long.

A Valid Password:
  1. Can contain any Unicode character.
  2. Is at least 6 characters long.
  3. Is at most 40 characters long.
  4. Must have at least 6 different characters.

 

The details of the business rules is not what’s at stake here. They should emerge out of the discussion of what is a Valid Username and Password, which is helped on the way by some examples:

 

Scenario Outline: Validating Username
    When I give a <username>
    Then The Account Application Service responds with an answer detailing whether the Username is <valid>.

    Examples: 
    | username:string                            | valid:bool |
    | user1                                      | yes        |
    | aVeryVeryLongUserNameWhichShouldBeAccepted | yes        |
    | æøåÆØÅ1234567890                           | yes        |
    |  user with space in beginning              | no         |

 

The examples here might not be that exhaustive, I’ve only discussed them with myself 😉 But the testing of this validation can be automated quite easily. And this will be the only Feature describing and testing the actual business rules regarding Valid Usernames and Passwords.

 

The Scenario Outline for Validating a Password would look just about the same. There should also be Scenario Outlines for Invalid Usernames and Passwords.

 

It is noteworthy, though, that the validation of Usernames and Passwords is not a User Story, it is – I think – more of an implementation detail. The same goes for the description of the REST Api.

 

For all the other tests (The Web Site, The App, REST Api) that need a Valid Username and Password (or invalid for that matter), it would be a good idea to have a TestHelper that supplies these, instead of dispersing the knowledge of Valid Usernames and Passwords over more tests than absolutely necessary.

 

I have chosen validation of Usernames and Passwords only as an example here. The business rules could have been anything else: financial rules, what ever. But where and how to describe the rules is still an issue.

Thoughts on usability

Designing software for ease of use is hard and as developers we seldom use our own software extensively when it has been delivered. But we are still expected to be able to put ourselves in the users seat and know how they work, to have the application work as seamlessly as possible and actually be a help in their work. This can be hard!

We are, however, not the only people in with this problem. To give some food for thought we can take a look on how ordinary house hold appliances work. I’m pretty certain that the typical developers of washing machines, tumblers, stoves and even to some extent TVs, are not the typical users of them at home.

Be aware of dangerous default settings.

I had a washing machine which, when turned on, had a default washing program where the temperature was 60 degrees (Celsius). If you weren’t aware of this you could easily ruin some clothes.

Don’t disable settings for no particular reason.

My current washing machine has a quick wash program, which takes 15 minutes on 30 degrees. But for some peculiar reason the machine can only sling the clothes at 1200 RPM instead of the maximum setting which normally is 1600 RPM. But on the quick wash program it is simply not possible to select higher than 1200 RPM. So in other words: I want my clothes washed in a hurry  but I have plenty of time to wait for it to dry?

Make sure that settings have obvious names.

My tumbler has two buttons, which – (probably inaccurately) translated from Danish – read: Delicate and Protect. And I cannot if my life depended on it remember what the difference between these two are. I’m sure at least one of them makes the tumbler dry the clothes with a lower temperature. But to figure out which one I have to look in the manual. Every time.

The user should be able to see what the application is doing.

My stove, which is a ceramic cooker, has some lights that indicate whether the cooking areas are “too hot” (to touch). But it has no indicator telling me whether any of the cooking areas are actually turned on – or off, for that matter. I have to manually inspect all the knobs to determine this.

Let the user multi-task if the scenario allows for it.

My LG TV has a Electronic Program Guide, which is fairly normal these days. But the brilliant designers has determined that I, when using the EPG, don’t want to listen to the program I was watching. The sound is disabled when I open the EPG. This seems like an odd decision (and it is very likely founded in something technical), since the sound would allow me to follow along in the program for some seconds, while reading up on what I’m actually watching 😉

 

These few examples show that we, simply by looking around, can spot usability problems. But how can we avoid them? Well, we will probably have to talk with our users about that… 😉

A passionate dream

I know this will sound a bit aloof, but I have a dream. A dream that software can change the World. Change it to a better place.

For software to be able to do this, we need brilliant and passionate developers. I am not a brilliant developer, but I am good and I am very passionate, which makes me more than halfway there.

How do you become brilliant? I’m sorry to be harsh, but if you are not passionate, you will never become brilliant. You can be good, even better than good, but to really shine you have to be passionate. The problem with passion is (as anyone who has been in love will confirm) that it can devour and engulf you. It will take an inordinate amount of the available time of your life. And you may – like me – feel that you never have enough time, because what you do is simply SO exhilarating.

Besides being passionate you should also be open minded. Never (hardly) think that you have the only right solution to a problem, unless you really have it (you just might). Do not be weary about standing on the shoulders of others. As Knuth said: “Software is hard”. And it is no shame to build your work on top of the work of other brilliant minds.

Building on top of the work of others of course requires that you actually know of the work of these others. Today it has become widely used to blog about your knowledge. And blogs are a valuable tool to find specific technical information. It has become a necessary substitute for the often more than lacking documentation that the various frameworks suffer from.

Reading books can be another very good way to learn. There is so much precious information in books – and at times not so much information as just pages. A lot of computer science theory has been described in books, so if you – for some reason – skipped the computer science classes, you may be able to teach yourself by reading books.

Education and training is an obvious way to learn. I fought my way through university to get a Master’s in Computer Science. To me it was rough, but I am glad that I did it. I try to keep up to date by attending training classes – not as many as I would like – they tend to be expensive. But I like going.

For me there is no doubt that I learn best by doing. No doubt at all. But I also very much like to theorize about a lot of things. I just like thinking about stuff, trying to work it out in my head. Quite often the real solution comes out in a somewhat different looking way.

One of the amazing things with the field of software is that there are so many sub-fields in which you can immerse yourself, which means that more developers than you may think will be able to find an area in which they can be passionate and hopefully grow to become brilliant.

Your way towards brilliance may be long and windy and full of detours. At times tiresome, at times fun and exciting. On the way you will learn a lot about what you definitely should not focus on.

I know for a fact that I should not focus on mathematically intensive areas. I also know that I’m weak in the language areas – especially functional languages, which I, for some reason, always have had difficulties wrapping my brain around. I would like to improve my skills here, though. And may even get around to be doing something serious about it. I did start on “7 Programming Languages in 7 Weeks”, but a bit more than 7 weeks have now gone by. Eventually I will return to it, but I must say that Prolog is hard. At least as difficult for me as functional languages, of which I have spent the most time on F#.

To be honest I am not yet very focused in my efforts, although I do have some areas of particular interest (not necessarily to be confused with actual endeavors): I like databases, especially SQL Server, and I have an interest in development tools – that is: tools that help developers in their efforts to produce the brilliant software the world needs to be changed. I might end up focusing more on this area, but then again:

There is just so much exciting stuff going on in the world of software.

EDIT:

Dr. Dobbs has two articles along the same lines:

What Makes Great Programmers Different?

What Makes Bad Programmers Different?