There are all sorts of reasons why using a functional programming language to build distributed systems is beneficial, but I won't go into details right now (maybe in a future post). For the moment, I'll relate what happened when I decided to brush up on my functional knowledge and then compare it to the imperative style.

I had learned Haskell in my university days as a young, wide-eyed undergrad, so I decided to start there and revisit it. To get back in the saddle, I decided to write a simple -- no, wait -- a very simple task manager.

Brushing up then going further

While developing the Haskell-based task manager (which I'd christened "Hask Manager"), I was reminded why the functional style appealed to me so much. Being a computer scientist by training, I see it as being closer to the theoretical underpinnings of computer science than the more popular imperative style is. I also believe the functional style brings numerous important benefits over the imperative style.

Detour: Remind me - what's functional programming?

In days gone by, when dinosaurs roamed the earth, these dinosaurs programmed in languages like assembly, Fortran or C. Writing an algorithm in languages like these is a bit like writing a recipe: explicitly prescribing each little step and each change in state to build up a whole series of steps (to be executed strictly in order!) and organise them into blocks called subroutines or functions. This is known as imperative programming and most of the popular languages in history have been imperative.

An imperative approach to doubling a list of numbers would look something like this (in Go):

// returns [2, 4, 6, 8, 10]
func getDoubles() []int {
	doubles := make([]int, 0)
	for _, n := range []int{1, 2, 3, 4, 5} {
		doubles = append(doubles, n * 2)
	}
	return doubles
}

But some other dinosaurs took a different approach called functional programming. Their approach was built around the notion of functions, although unlike "functions" from other languages like C (which are really procedures), their functions were based on the mathematical idea of a function: an expression that has no state or mutable variables, and has no side effects when run. Every time you execute a function with the same set of arguments in such a language, the result is always the same. These ideas developed into languages like LISP and Scheme.

A functional approach to doubling a list of numbers would look something like this (in Haskell):

-- returns [2, 4, 6, 8, 10]
getDoubles = map (* 2) [1, 2, 3, 4, 5]

Today, there's not such a hard distinction between functional and imperative languages. As time goes on, imperative languages incorporate more and more ideas from the functional approach.

Back to the story...

At this point, I was only playing. I had no grand aim in mind for this little side project and so it grew organically as I continued to play. Once the Haskell task manager was up and running, I decided to try and compare the functional style of Haskell with a suitable non-functional language. I decided to rewrite Hask Manager using an imperative language instead. For this, I chose the Go language (Google's flagship programming language) and this version of the task manager I called "Go Get Things Done".

Go logo by Renée French

I had no strong reasons for choosing Go, but a few things made it feel suitable:

• I'd been learning and using Go for about a year (I've made some contributions to Kubernetes and CoreOS, which are built using Go).
• Not only is it imperative, but it avoids a lot of functional ideas (although it has taken a few here and there).
• It's a young language which means its designers had a lot of existing software engineering wisdom to draw on. The Go authors have explicitly stated that they are quite ruthless in deciding whether a feature should be implemented in the language. If they've being choosing wisely, it should be a language with no outdated, legacy concepts. This makes it a "worthy" object of comparison.

What did I end up writing?

Both versions of the program are functionally the same. Nevertheless, I tried to stick to each language's best practices as best I could. The resulting program is really simple. It understands commands to maintain a task list, such as:

• all
• add <description> <date>
• todo
• today
• show <id>
• tick|untick <id>
• quit

Both versions are available on my GitHub account:

• gogtd
• Hask Manager

What are these STOPs in the code?

As I was familiarising myself with Haskell, and later when I was comparing the two versions, I wrote comments in the code. I have a leaky memory, so I find writing stuff down is the best way for me learn and have it stick.

In case anyone else is interested in learning the basics of either language, or just looking through the code comparison, I put the phrase "STOP" in each comment and a number denoting the order in which I recommend you follow them. So, if you open the repo in an IDE and search for the phrase "STOP", you should get a nice convenient list of stops to go through.

Interesting notes

In no particular order, some observations I had as I was writing these programs.

Learnability

I believe functional programming is harder and takes more investment to learn than the imperative style.

Experienced programmers who already know the imperative approach but not the functional will have a lot of assumptions and deeply embedded habits that will need overcoming.

But I suspect a novice would fare little better if they learned the functional style first. Functional programming involves some quite abstract mathematical thinking that is not highly intuitive for a beginner. Conversely, imperative programming is more about small concrete steps. All in all, a beginner can grasp an imperative language quicker and achieve something in it after less investment.

Finding mistakes

One thing that makes Haskell harder to learn is the abstract error messages which I found difficult to decipher when learning. However, once I got the hang of things, I found that the time spent understanding and correcting errors in my programs became much shorter.

I even got the feeling that, with experience, correcting errors in the Haskell code took less time than the Go code. In Haskell, once an error was understood, the fix was usually clear. In Go more time was spent running through a piece of faulty code procedurally, trying to work out the problem and trying multiple changes to fix it.

In fact, the rather unsettling "works after first attempt" (that frightening occurrence when the first draft of your code works without a problem) happened a few times after writing a Haskell function, something that I find happens a lot less often in Go (or any imperative language).

Functional as a prototype

I've read online a few examples of people developing a Haskell version of a program as a prototype before then re-implementing it in a more 'mainstream' language. (I've also read examples of people finding that their Haskell prototypes worked just fine, thank you, and subsequently junking plans to re-implement.)

If that were your desire (although I think Haskell is a perfectly suitable implementation language), I see the advantages of the approach. I noticed a few places where the Go version of the task manager took inspiration from the Haskell version and was an improvement over the version that I probably would have written.

One example is the commands.go file, which is a sort of backend for updating task data. I can imagine that I would have written the functions in there as less generic versions (e.g. GetTodaysTasks or GetUndoneTasks) had I not first written a Haskell version which, by the language's nature, encouraged me to write the functions generically. That backend turned out more like a database API, with a small number of generic functions (AddTask, QueryTask, UpdateTask etc.). This meant that the backend could stay concise and reuseable, and still allow the programmer to easily come up with an endless variety of different operations by filling in the blanks of the generic functions. However, the Go syntax makes this same functionality less readable than its Haskell counterpart.

// main.go
if strings.HasPrefix(input, "todo") {
	return gogtd.CmdQueryMany(
		func(tl *gogtd.TaskList) *gogtd.TaskList {
                        return tl.Filter(gogtd.IsPending)
                },
	)
}
 
// commands.go
func CmdQueryMany(f func(*TaskList) *TaskList) string {
	tasks := GetTasksFromFile()
	resultSet := f(tasks)
	return resultSet.String()
}

--- Main.hs
interpret input
    | "todo" `isPrefixOf` input = cmdQueryMany getPending
 
--- Commands.hs
cmdQueryMany :: (TaskList -> TaskList) -> IO String
cmdQueryMany f = do
    tasks <- getTasksFromFile
    return showTaskList(f(tasks))

Making Go more functional

Just because support for the functional style is limited in Go, doesn't mean you can't add a little bit yourself. For example, the standard functional primitives (map, filter and reduce/fold) may be missing, but with first-class functions in Go you can roll your own (see Filter in task_list.go).

func (tl *TaskList) Filter(f func(t *Task) bool) *TaskList {
	newTl := NewTaskList()
	for id, t := range tl.Tasks {
		if f(t) {
			newTl.Tasks[id] = t
		}
	}
	return newTl
}

Of course, this example not as powerful as filter from Haskell, because it can only be used to filter a TaskList (whereas filter in Haskell is generic).

Conciseness

The Haskell version is more concise, but this is hardly surprising. Functional programming is generally thought of as being more expressive, allowing you to achieve more per line of code.

Side effects in Haskell

Once you introduce a side effect into one function, it can seep into other parts of your program. Just by using such an "impure" function, another function allows side effects to influence its behaviour.

You can write functions with or without side effects in both Haskell and Go. A key difference between Haskell and a language like Go is that Haskell is explicit about which functions deal with side effects. Once a function has allowed side effects in, it must be marked as such, and that "bubbles up" any other functions that use it.

Safety in Haskell

Similarly to its strictness in dealing with side effects, Haskell is also strict when dealing with potential errors. If your function calls another function that might be unable to return a value (via a Maybe) or might return an error (via an Either), you must deal with it. There's no getting away with ignoring potential error values as there is in Go.

Summary

That's all for now. I hope in the future to do some more stuff along this theme... maybe writing a comparison that makes use of concurrency features, explaining in greater detail how functional programming works, or expanding on how it can be used in distributed systems.

Watch this space.

This is a quote from a great article called Why the Great Glitch of July 8th Should Scare You by Zeynep Tufekci. You should go and read it (but finish this one first).

It's one of several articles that I've noticed recently explaining the sorry state of software quality. It's nice finally to see some prominent articles emerging which put the record straight. I'm glad a few people are doing their best to reveal the dirty secret of programming: software mostly sucks. It's full of bugs and it's insecure.

This article (and others besides) go into great detail. If you want the quick and easy read, here's my attempt:

Speaking as a software engineer, the problem is:

• We write huge amounts of software and software is extraordinarily complex by nature.
• At the same time, our software has to integrate with an even greater mass of existing software, which is made up of programs layered on top of one another most of which also barely work.
• And all the time we are pressured into doing this as quickly as we possibly can.

It is possible to do software well and write programs that are stable and secure. For example, read how NASA do it. They work with extreme care at a relatively glacial pace; single changes to the code result in committee meetings and can throw up specifications running to hundreds of pages. But it's the kind of working atmosphere most programmers and start-ups would baulk at. (From the article: "The culture is intolerant of creativity, the individual coding flourishes and styles that are the signature of the all-night software world.") That kind of quality requires spending a lot of money. Of course, NASA, with their huge government budget, can do that. The rest of the world has to turn a profit.

So software sucks - and it'll probably stay that way for a good long while, not because of any technical problem, but more a cultural one. Fast, cheap or high-quality? You can pick only two out of those three. Guess which two our culture ends up choosing time and again...

We judge FLOSS essential for several reasons:

• We know that FLOSS is critical for a society like ours, one that is dependent on information technology. In a software-powered world, ensuring that software is a common good for all (and not a secret in the hands of a few) means that people are empowered by technology and not restricted by it.
• What's more, we at Endocode all come from backgrounds in FLOSS. Each one of has for years been involved with various open source communities and have come together in a company that provides multiple services to numerous organisations - services built on the power of open source software. Without FLOSS, Endocode wouldn't exist.
• And that power translates into altogether more practical reasons for valuing FLOSS. We know that being able to examine and adapt software gives us the power to do more for our customers and ourselves. We know that FLOSS helps us to reduce costs and increase quality. We see using open source software as the superior way to build large, complex and robust systems.

The FSFE fights many campaigns, such as promoting awareness among the public, pushing for FLOSS and open standards in the public sector, and defending us against menaces like software patents.

Endocode has become a silver sponsor of the FSFE. In so doing, we recognise their contribution and join the many other individuals and organisations (like HP, Intel  and LibreOffice) who want to see the FSFE continue doing its great work.

But it wasn't just plain old Java; the participants were already well-versed in that. Instead, they wanted to learn how to build distributed systems using Java. Not only that, but they had a few very specific interests, which you see in the list below.

Distributed systems is a challenging topic to teach, especially because it encompasses so much stuff and I had only two days to cover the requested material. Here's what I managed to include:

Principles of networked and distributed systems

• Definitions
• Transparency
• Benefits and challenges
• Case study: The web

Distributed architecture and design patterns

• Layer paradigm
• Distributed operating systems
• Interprocess communication
• Remote invocation
• Models (client/server, peer-to-peerm multi-server, proxy, mobile variants, thin clients)
• Distributed design patterns

Threads

• Thread vs. processes
• Benefits and costs
• Server threading architectures
• Threads in Java

Lower-level concurrency mechanisms

• Thread interference
• Synchronising

Higher-level concurrency mechanisms

• Locks
• Executor, ExecutorService, ThreadPools
• Futures
• Concurrent collections

Sockets

• Addressing with sockets
• UDP
• TCP

Streams

• Input and output streams
• Stream readers and writers
• Streaming with sockets
• Streaming among multiple threads
• Multicast

Serialisation

• Serializable interface
• Serialisation with references
• Serialisation with inheritence
• Best practices

Remote Method Invocation

• Using the RMI registry
• Remote interface
• Creating a server
• Creating a client

Case study: CORBA

Distributed object persistence

• Java Persistence API
• Hadoop
• Distributed databases
• Special topic: Distributed cache using memcached
• Best practices for distributed caching

Of course, it's very nice for someone to pay your work compliments, like being called "eloquent" and having an "easy, engaging style". But there are other things in the review which I'm particularly pleased to read because they show that I'm achieving my goals for the book.

For instance, the reviewer agrees with me that the book is "aimed squarely at the intelligent layperson, it requires no prior expertise and sits within the genre of popular science." I'm glad that I have managed to present these ideas in an understandable way that requires no background knowledge.

Furthermore, the reviewer recommends the book to target audiences that I also intended to shoot for: "IT professionals, teachers, parents and their teenage children will all find it an invaluable introduction to the key concepts and their practical application." This is especially nice to read as I now know that the reviewer is in the field of education, working at a British school and active in the Computing at School BCS working group.

In the reviewer's opinion (and mine too) Brown Dogs and Barbers is also a book that's relevant to people already working in IT, stating: "If you have no background in computer science, this book will be a revelation. And if you think you know what computer science is about, this book will invoke connections you may never have considered before."

If you'd like to read for yourself what prompted this review, you can order my book online at Smashwords or Amazon, where there are also samples to try before you buy.

In recent years - but at least as far back as when I was a pupil in the 1990s - education in computing and computer science within British schools had a rather narrow focus. Children learned mainly about operating computers: using word processors to write documents, whipping up spreadsheets, (maybe) building simple databases and proficiency in using an operating system (any operating system, so long as it's Microsoft Windows).

There's nothing wrong with this. It's a fine goal to teach someone how to make good use of everyday applications. However, it must be admitted that this narrow focus merely teaches children how to be passive users of a computer. It gives them no grounding in the fundamentals of computing; they learn nothing about how a computer actually works.

But the upcoming overhaul of computing education will change that. Computing education will in the future focus on things like what an algorithm actually is; how to program a computer; how a program relates to an algorithm; how to detect errors in programs; how to reason about source code and find errors, and much more. It will be like physics lessons going from focusing on how to drive a car to learning the principles of the internal combustion engine.

And these changes won't only affect college-level, or secondary school-level. It will begin from the first year of primary school.

Parents naturally want to support their children's learning at home. With many subjects, you can do this. Many of today's subjects are the same as when you were at school (Maths, Science, English, History etc.), so discussing their contents and helping with homework are doable. But chances are you were taught nothing about computer science at school, so how could you support your child in this subject?

One way to get a feeling is to look at the proposed syllabus. Schools in England and Wales divide all schooling into several blocks called key stages.  Each key stage covers several years of a child's education.

Key stages 1 - 3 cover all of primary and most of secondary education. Children educated within these stages are aged between 5 and 14 years. Here's a link to the UK Government's breakdown of plans for teaching computing in England, but I've picked out some of the key parts here:

Key stage 1 (aged 5 - 7 years)

At this stage, some things your child will learn include:

• What an algorithm is
• What a program is and how it relates to an algorithm
• What a digital device is
• How to perform simple programming

Key stage 2 (aged 7 - 11 years)

• Understand the importance of sequence, selection and repetition in algorithms
• Understand computer networks
• Use logical reasoning
• Understand and detect errors in programs

Key stage 3 (aged 11 - 14 years)

• Use and evaluate computational abstractions
• Understand key algorithms (e.g. for sorting and searching)
• Understand binary numbers and their use in computing
• Understand how instructions and data are stored and executed in a computer

Some may look at that list and find that most of the items mean nothing to them. That might be discouraging if you're a parent with a child in school. Nevertheless, it might prompt you to learn about the subject for yourself so you can share in what your son or daughter is picking up in computing lessons, but you may be unsure where to begin.

That's one of the reasons I wrote my recently released book about computer science, Brown Dogs and Barbers. It has several intended audiences, but one of the primary ones is those people with no background in computing whatsoever who would like to learn about its fundamentals. That's why it's an easy-to-read book with a fun, casual style and touch of humour mixed in.

As an indicator of how helpful Brown Dogs and Barbers should be, compare the list of topics covered in the book (below) with the school syllabus. Topics that appear in both the book and the syllabus are emphasised:

• Graph Theory
• Set Theory
• Sequence, selection and iteration
• Algorithms
• Theory of Computation
• Turing Machine
• Halting Problem
• Complexity Theory
• Binary and Hexadecimal
• Binary Architecture
• Von Neumann Architecture
• Machine Coding and Assembly Language
• High-Level Programming Languages
• Searching and Sorting
• Data Structures
• Multi-tasking
• Scheduling
• Concurrency
• Operating Systems
• Networking
• Security

I think that my book is ideal if you have school-age children and want to brush up on computer science so that you can prepare yourself to help them get to grips with this sometimes challenging but nevertheless rewarding and important subject.

It's available to order at Smashwords or Amazon, where there are also samples to try before you buy.

Right now you can get it from several distribution channels, including Amazon (find it your nearest Amazon outlet, like the US, Canada, UK or Germany) and Smashwords. Other retailers, like iTunes, are also currently preparing it for sale in their webstores. More news on those as I receive word.

We live in exciting times. My book, Brown Dogs and Barbers (which explains computer science to just about anyone who can read), is very close to publication.

The funding drive over the last few months raised enough to produce a professionally designed paperback and ebook, complete with crisp design, beautiful diagrams and an insanely cute front cover. It will be available to buy in places like Amazon and iTunes later this month.

Until then, you can follow the links above to get hold of a sample of the final book...

... and behold the front cover!

I'm not certain yet, but I think PayPal have recently made changes that have stopped the button from working... I'm currently looking into getting it working again.

In the meantime, if you're interested in using PayPal to donate to the production of my book, you can send contributions to my email address: [email protected].

Thank you for your continued support.

I'm pleased to report that with your help the total recently went over €1000 - in fact they currently stand at €1085. This not only means I can now hire a proof reader and an artist, but I am almost at the point where I can commission the production of a paperback version that can then be put on sales in places like Amazon and Lulu.

I'd like to thank everyone who donated so far. There's still more I'd like to do, so please either donate to the project or pass this information on.