clock

Software development and your natural rhythms

General software engineering, , , , ,

One of the lessons from Joel Spolsky I took to heart right from the start of my career as a software developer was to schedule tasks honestly. Scheduling dishonestly means estimating a task when you don’t really know what work is involved. As a result, you end up putting an incorrect time estimate on it.

To combat this, you should schedule honestly and Joel proposes this simple rule: tasks should be measured in hours not days, with a realistic estimate being no larger than 16 hours. If your estimate is larger than that, then you probably don’t appreciate the true size of the task. Experience tells us that programmers usually underestimate tasks they don’t fully understand, so estimating in terms of days or weeks risks schedule overrun. Tasks which will take more than a day or two should therefore be broken down into smaller sub-tasks, because this forces you to think about the work in more detail and gain a better understanding of it.

But how much time is in a day?

So an 8-hour task will take about a day, yes? Well, maybe not. One of the other things I recall Joel saying is that there is less time in the day than you think. Yes, we ‘lose’ time to lunch breaks and meetings, but that’s quite easily taken account of in a schedule.

What about that other ‘lost time’? You know what I’m talking about: all that web-surfing, checking email, chatting at the water-cooler. How much time every day is ‘lost’ to these little pleasures? And how guilty do they make us feel when work ends, because we know we spent a cumulative hour on Facebook instead of completing that task which should have taken only a day.

In actuality, we probably shouldn’t be feeling guilty about this at all. We should just be scheduling better.

Scheduling in cycles

I recently read this article: “The origin of the 8 hour work day and why we should rethink it“. It presents interesting arguments for switching away from a traditional 8-hour work day and has several tips for working more effectively. The one that really intrigued me was the argument that human concentration follows a natural rhythm whereby we can focus on a task for a certain amount of time before we start to lose it and need a break. The article claims that most of us can hold focus for about 90 minutes before requiring a break of around 20 minutes, and so advocates planning work time in 90-minute windows (let’s call them cycles).

Ultradian Rhythm of Life

So what happens when we combine this with Joel’s advice? A task estimated to take 6 hours effort (360 minutes) would require 4 cycles (4 x 90 minutes). When you factor in the 20-minute breaks between tasks that’s an extra hour.

If you can manage more than four cycles in a day then good for you, but four is enough for me. Under this cycle-scheduling system then, “one day” (if we stick to an 8-hour day) clearly gives you six hours work time, a one-hour mealtime, and recognises that we all give in to the temptation to take little breaks now and again.

Can it work for programmers?

Let’s consider a few things.

  • It factors in those more unpredictable distractions (Facebook, email, coffee etc.) which are actually symptoms of us reaching a loss of focus. Using cycles allow you to actually allocate time for them, which recognises we need them and stops us from feeling any guilt about ‘wasting’ time.
  • It can be encouraging to know that your current cycle of work will soon be followed by a scheduled break, especially when the work is intense.
  • You can more easily track the work you did. At the end a day, can I really be sure I put in X hours of work? How much cumulative time was lost to those little breaks? If you follow the cycles, you have more confidence in how much time you really spent on tasks.
  • Granularity. Most of the time, programmers can’t measure progress with the granularity of certain other professions (like a bricklayer who can measure bricks laid per hour). But there are still activities we can plan which fit into a 90-minute cycle. Think along the lines of: layout one basic screen design, implement that setter function, write the code that makes TestX pass. And speaking of testing…
  • Cycles can also help to make the distinction between coding and testing clearer. When you estimate a task as 8 hours, it’s easier and tempting to lump the two together in a schedule only to find that coding takes up all the allocated time and testing gets pushed back or even dropped. Using 90-minute cycles encourages the approach of spending one cycle getting something working and then spending the following cycle reviewing and testing what you’ve done. Most programmers can relate to how different code looks when it’s examined again after a break and how this makes the flaws easier to find.
  • Interrupting the flow. One objection I’ve heard goes along the line of: “It can take programmers several hours just to get into a task, so you shouldn’t interrupt the flow.” This is true, but getting into a task still needs factoring into a schedule somehow. Just like cycles force you to think about the actual small details of a task, they also force you to plan your research or preparation.
    • Try reducing the breaks in-between preparation cycles to 5 minutes.
    • Be more detailed about what “getting into a task” means: How many pages of a resource can you read in one cycle? How much of a prototype/test code can you write in one cycle? etc.

Summary

So, instead of estimating a task or feature in terms of the number of hours it will take, why not in terms of the number of cycles instead? Personally, I like the approach and I already work in a similar pattern anyway. If it is indeed effective and quite natural to all of us then why not  make scheduling explicitly take it all into account and give yourself a more accurate project plan?

GNU

Happy Birthday GNU!

News, ,

It was 30 years ago today that Richard Stallman announced something kind of crazy: he was going to initiate a project to implement a version of the Unix operating system that was completely free (as in freedom). He called the system GNU (GNU’s Not Unix) and his original announcement is preserved here as a piece of IT history.

GNU is highly significant for FLOSS fans. It was the first project that consciously and clearly set out sharing of source code as a basic principle. The Free Software movement didn’t yet formally exist in 1983, but Stallman went on to become a key figure in its formation as time went on. GNU itself became a key component of the Linux operating system.

If you want to learn more about GNU, go over to gnu.org and read up on its history and the philosophy behind it all.

Bruce Schneier

Bruce Schneier and the Lords of the Cloud

Uncategorized, , , ,

I know the title sounds like a Harry Potter rip-off starring everyone’s favourite IT security guru, but this is a video of Mr. Schneier giving a talk at Google about the state of security and privacy online.

I’ll embed the video below (I highly recommend watching it). For those who don’t have an hour to spare, I took some notes of the most pertinent things Bruce said.

 Notes

  • The Internet (like most technology) magnifies power – those who already have power become even more powerful.
  • Users are becoming less powerful with the development of things like cloud computing and the removal of certain functions  like the ability to clear cookies and access the file system.
  • Security on the Internet is becoming like feudalism: the Lords of the Cloud (Google, Dropbox, Microsoft etc.) have the power and promise to protect us if we give them our allegiance.
  • The Internet (again, like most technology) is a double-edged sword — there are two sides to every aspect: one positive or benign, the other negative. Consider:
    • content-filtering vs. censorship
    • marketing vs. propaganda
    • central quality control (Apple) vs. central government control (China)
  • Surveillance is the business model of the Internet.
  • Schneier sees corporate and government power (in the US) as aligned. Each group uses the other to get around the restrictions law places on them (e.g. the government could never pass a law to make people carry tracking devices… but we all carry smart devices; corporations use the government to pass laws that protect their business models).
  • It’s an open question whether the Internet changes the balance of power in favour of the small and powerless or not.
    • But it’s notable that the powerful are often winning the arguments in recent days.
  • We need more research into surveillance, propaganda, censorship and use control online.
  • We need to ensure safe places for anonymous publishing (like Wikileaks).
  • Making circumvention illegal and abolishing net neutrality allows the powerful to become even more powerful.
  • The Internet is not a public sphere. It is a collection of privately-owned public spaces.
padlock

How being open with your data keeps it safe

General software engineering, , , , , , , , , , , , ,

In a previous post, I gave some examples of FLOSS programs that performed essential computing tasks (namely operating system, email and cloud storage) which you could use instead of proprietary alternatives. I claimed that the FLOSS versions had the potential to improve the security of your data (in case you’re concerned about unauthorised third-parties accessing it) without going into detail about how.

This post will explain why entrusting your data to open source software can protect it from snoopers. Please be aware that there are many ways to spy electronically; I’ll only be addressing some that the use of FLOSS can guard against. Also be aware that computer security is not my speciality, I’m just trying to present the issues as best I understand them from several perspectives.

The “Many Eyeballs” Perspective

EyesWhen I discussed operating systems in my previous post on this subject, I pointed out how it’s hard to know exactly what a closed source program does. This has a lot to do with how software is built, which I’ve explained before: programmers write software in a human-readable source code and then run that code through a compiler to produce the binary program that’s readable only by a computer. While closed source (a.k.a. proprietary) software is released in binary form only, software released under a FLOSS licence comes with the source code as well, a bit like getting a Haynes manual (those trusty books that detail every single component of an automobile) supplied with your new car. Conversely, with a proprietary program, you get the equivalent of no manual and a car with a sealed bonnet.

Without the source code, you’re forced to look at the program like a black box, so all you can see is its external behaviour. This can make it extremely difficult  to determine its internal workings. You may wonder whether there’s any code in the program that makes it “phone home”, reporting information about you to a remote server. You may ask yourself if there are vulnerabilities in the program which would allow someone to log into your system without your knowledge. For questions like these, there’s no easy way to find the answers without the source code.

With the source code, you have the opportunity to search for such vulnerabilities. When many people are able to look through the code, it should go without saying that all these eyeballs act as strong defence against the insertion of secret spying routines and hard-coded back doors. This is not just all theoretical speculation. In the 1990s, for example, Borland released a database server called InterBase that had a back door intentionally engineered into it (see David Wheeler’s article mentioning this), which allowed people with the knowledge to break in. As long as the program remained proprietary, this vulnerability remained secret (at least six years).  After InterBase was eventually open sourced, the back door was discovered within months.

Bruce Schneier

Bruce Schneier

As well as gaining access to the program, the availability of source code helps you verify what happens when sending stuff out from the program. Whenever you want to transmit data securely from your computer, you need to encrypt it using an encryption algorithm, which turns your readable data into “meaningless” cipher text. There are many such algorithms available, some strong, some weak. With access to the source code, you can verify for yourself whether or not the program has poor quality encryption routines and thus makes it a simple matter for third parties to decode your intercepted messages. It’s partly for this reason that famed IT security expert Bruce Schneier recommends that security-conscious engineers use FLOSS.

Keep in mind that the benefits of access to the source code are not automatic. Just because it’s possible to review the code for vulnerabilities, that doesn’t necessarily mean it gets done. The community that develops the software needs to be sufficiently large, active and knowledgeable about IT security.

The “Keys and Padlocks” Perspective

What I termed the “Many Eyeballs” Perspective is worth knowing, but from what we know about the current spying scandal it isn’t actually the most relevant one. The stories that have dominated the front pages for most of this summer are not about secret source, but rather who has the keys to your information.

One of the most important concepts in contemporary computer security is public key encryption. It’s a bit complicated, but it’s basically a system that simulates keys and padlocks in software. Each user has a public key and a private key, which are both actually just very long numbers in a file and are mathematically linked with each other. The public key can be passed around freely to anyone you like, whereas the private key must be kept secret. Although it’s called a public key, by analogy it’s more like a padlock.  I’ll explain…

The classic application of this idea is message encryption. If your friend wants to send a message intended only for you, they have to encrypt it so anyone who might intercept it is unable to read it. Encrypting a message is easy: your friend just uses your freely available public key to turn the plain text into cipher text. However, decrypting the message can only be done with the corresponding private key (hence why you must keep this one secret). By analogy, it’s as though your friend wants to send you a box and be certain that only you can open it. You send your open padlock (public key) to your friend, who then uses it to seal the box. Only you have the (private) key for this padlock, so your friend can send the box knowing no-one other than you can unlock the padlock if it’s intercepted. If you want to keep your emails safe from prying eyes, encrypting them prior to dispatch using algorithms such as PGP is always a good idea, but there’s another way to apply this idea other than encryption, and that’s authentication.

Authentication deals with who is allowed access to a computer system. The most common way of logging into a computer (i.e. being authenticated) is with a username and password. However, passwords are sometimes unreliable, particularly when the password owner is sloppy. Choosing the name of your pet cat as a password is pretty easy for a nefarious hacker to guess and a brute force attack can try every word in the dictionary in a very short time. (Hence the reason why your system administrator forces you to use passwords with a minimum of 32 characters and utilising letters, numbers, mathematical symbols and Egyptian hieroglyphs.) An alternative is to use public and private keys. Instead of asking you for a password, you give your public key to the server administrator, who puts it on the computer, while your private key remains on your own computer. Thereafter, when you attempt to log in to that server via your computer, the server checks whether the public key and private key match. If they do, you are granted access to system — and all without a single bloody password!

Your public key might be the only one on the server or it could be one of many, it makes no difference. When your public key is added to a server, it’s as though a new door into the system is built and your padlock is used to lock that door. This gives you your very own exclusive entry point. There could be dozens of other doors, but you can only open your own; anyone without a door has no way to get into the system.

Now, here’s the rub. The person in charge of the system gets to set up the doors. If you are the system administrator, then you know who has access to your system. However, if responsibility for the system is delegated to an external party (as is the case with most popular web services we all use) it can be really hard to know for sure who has access. They own and maintain the servers not you. Of course, there’s a secured door that only you can pass through, but how do you know whether or not other secret entry points exist, the so-called “back doors” that let someone else access your data?

KolabThis is pretty much the situation with many web-based services like GMail and Dropbox. They stand accused of setting up back doors on their services which allow intelligence agencies to step in and poke around at will. Sadly, I’m afraid there’s not much you can do about this. They certainly won’t provide you with a copy of their software which you can use to set up your own email servers or cloud-storage services.owncloud

But you do have a choice with FLOSS programs, which is why (in the previous post) I recommended Kolab and ownCloud as open source alternatives to their proprietary counterparts. By deploying these programs on your own server, you get to exercise control yourself and prevent the installation of secret back doors.

Bonus Perspective: “The Wiretap”

(This last perspective follows on from the previous one, although is not exclusively related to FLOSS. )

Another aspect to the spying scandal is the alleged listening in on communications as they whizz through the Internet, essentially wiretapping the world wide web. If you’re concerned about the security of your messages, you should encrypt your outgoing traffic with encryption methods like PGP. With FLOSS programs this is usually quite simple to set up and, as a bonus, you have the chance to control the encryption keys.

However, proprietary programs don’t always make it easy to encrypt traffic, or they might deny you the option altogether. (Just try and encrypt your GMail traffic and see how difficult things get.) What’s more, where traffic is encrypted, the service providers may be in control of the encryption keys and we’re back to the problem in the “Keys and Padlocks” perspective: how far do you trust them not to reveal the keys?

Conclusion

As I’ve shown over these last two posts, FLOSS-based programs have the potential to improve the security of your computer-based data and keep prying eyes away. An operating system like Linux, email services with Kolab or cloud-sharing via ownCloud are all safe bets.

But I must stress the benefits are not automatic. There are prices to pay.

For one thing, more proactivity is required. Of course, software developers need to be well-clued up on writing secure software, but privacy-concerned users also have to be vigilant. They have to take on more responsibility for the data and ask the right questions, like: Is this software FLOSS? Who is responsible for controlling access to my computer and its programs? Is proper encryption being used? etc.

facebook-pigs

Also, there may be a price to pay in a more literal sense. A famous comic depicts two pigs chatting with each other, remarking how lucky they are to be living on a farm that feeds them and houses them — all for free. Too good to be true? As Georg Greve points out, when you’re the product being sold, being put under surveillance is part of your payment to the service provider. I will venture to say that when you are instead the paying customer of a service provider, particularly one that uses FLOSS, surveillance is probably a much lesser risk… but still don’t forget to read those terms and conditions.

In short, paying those prices gives you the chance to be an empowered customer rather than a product sold at the market.

padlock

FLOSS alternatives for the post-PRISM era

Opinion, , , , , , , , , , , , , , ,

In this post, I’ll describe a few alternative FLOSS programs for any proprietary software you might be using to handle your sensitive data. Exactly how FLOSS makes life harder for peeping toms I’ll describe in a follow-up post.

(Edit: You’ll find  that follow-up post here.)

Background

In recent months we’ve learned about how the national security services of various countries have been spying on their citizens — and using the citizens’ own computers to do it. What’s more, the largest IT companies (household names like Google and Microsoft) have admitted to assisting the security services in their quest to monitor everything we do online. Understandably, many people don’t approve. But hey, what can you do? If the government wants to break into your computer or listen in on your Internet transmissions, is there really anything you can do to stop them?

I suppose espionage has been with us so long it’s just a fact of life — after all, some call spying the second-oldest profession — so it won’t be going away any time soon. But if you’re concerned about your data being available to anyone who cares to try and grab it, using FLOSS (Free, Libre or Open Source Software) offers a way to combat snoopers and eavesdroppers.

Why not consider the following alternatives?

Operating System: Linux

The operating system is the ultimate master of a computer. It manages everything your computer does at every level and so is responsible for authorising access. This makes the choice of operating system a critical part of IT security.

Tux: The Linux Mascot

Tux: The Linux Mascot

It therefore pays to know what an operating system is doing. Most computer users today have some form of Microsoft Windows installed on their machine. But, since Windows is a proprietary program (i.e. you can’t read the source code) it’s extremely difficult to know exactly what Windows is up to. Is there a little piece of functionality somewhere that secretly opens up a back door or reports some unauthorised information back to a server somewhere? It’s exceedingly hard to tell.

With a FLOSS operating system like Linux, such a question is easier to answer because all of its source code is publicly available to read. Yes, you personally might not be able to read computer code, but there are thousands of coders who routinely read through Linux’s source. Assuming no vast conspiracy exists among these largely voluntary programmers, we can imagine that the whistle would be blown pretty quickly if someone tried to sneak some spy-friendly routines into Linux (an example of which occurred in 2003).

Still, there are many flavours of this flagship operating system to choose from. Even in the world of Linux, privacy issues turn on your choice of flavour. Ubuntu Linux for example, arguably the most popular and user-friendly, recently rubbed many users up the wrong way after introducing a “feature” that automatically shares some of your information with Amazon.

Email: Kolab — Cloud Storage: ownCloud

What can you do to stop people reading through your mail? There are actually two main issues here. One is related to what I’ve already been talking about: namely, if the source code to your email program is a secret, you can’t know for sure what it’s doing.

KolabThe second is: where is your data? In today’s world of the cloud, your data — whether it’s emails or files shared via a service like Dropbox — is probably stored on some anonymous, remote server thousands of miles away rather than your own machine. Someone else is therefore managing access to your data. In such cases, as recent allegations have suggested, it is possible for them to grant access to people whom you know nothing about.

For email and cloud storage, I nominate Kolab and ownCloud as alternatives respectively. Kolab (as I’ve written about elsewhere) is an email service that prides itself on being built from 100% free software. OwnCloud is a file-sharing web application whose entire codebase is available on GitHub. So how can you get hold of them?owncloud

In both cases, professional IT companies host the application on their servers and you can set up an account with them and use the software. Admittedly, just like with GMail or Dropbox, you’re taking the word of these organisations that your data is safe with them. But with Kolab and ownCloud, you’re not forced to trust such companies. If the want to take greater steps to protect your data, then you can do something that isn’t possible with Google and Dropbox: you can set up your Kolab email or ownCloud storage on your own server and thus empower yourself with responsibility for your data. Many organisations are beginning to do this, thus taking on the responsibility and becoming the sole gatekeepers to their own data. What’s to stop you doing the same?

VoIP: ?

The other great revelation from the PRISM scandal was the collection of data from Skype calls. Skype is one of the most popular VoIP programs and I’ve seen it used regularly by universities, companies large and small, as well as private users by the million. So what FLOSS alternative exists as a more secure alternative?

To be honest, I’m not sure. Alongside the operating system, email and cloud storage, VoIP is one of the most important components for addressing IT privacy. But I also think it’s one of the weakest sectors in which FLOSS competes. I’d love to hear recommendations and success stories in the area of open source VoIP, but sadly even my very geekiest of friends use either Skype or Google Hangouts.