Home » process

Tag: process

Innovation is messy

A hard lesson I have learned recently. 

I like to be right. I like to be efficient. I like to do things the correct way the first time around. I would like to believe that my experience would allow me to do so. 

But some projects present an interesting problem. To understand the problem, let’s imagine we are the architects of a tall and spectacular building

We know how the building is supposed to look. We know what purpose it will serve and who will use it. With this information, we can start making our plans starting from the ground up. We design a solid foundation, and then we layer on top of that floor after floor until we finish. 

However, this is not innovation. Is following a well-established workflow where there are little to no unknowns. We can make good decisions about what materials to use where and a reasonable estimate about when the job will be done. We don’t expect many surprises along the way. 

But what if we have this idea to use a new material, and design this building to serve some innovative purpose that no one has done before. Now there is no way to lay down a solid foundation because you cannot answer the question: “solid for what?”. 

You may discover halfway through that most design decisions do not help you achieve your vision due to some unknown limitation that was invisible right until you got to this point. So you have to dynamite the whole thing, learn your lessons and try again. 

Large, innovative software projects are like that. The architecture you started with may have looked great in the beginning but ends up feeling very limiting when you suddenly realize you need to make a dramatic shift in your project, and your “foundation” does not allow for it. Tearing down a software project is free, compared to dynamiting a building, but you still won’t get your money back from all the work that you cannot use anymore. 

But not all is lost. Because in this process of trying and failing, you learn and you grow into your idea. You stumble into the things you didn’t know that you didn’t know. And drip by drip, you make the unknown, knowable. 

This “failing often” is a challenge for me to accept and work with because it feels wasteful. In hindsight, “I could have done better!”. But thinking like that is a trap, and it suffocates the very creativity required for innovation. You need to be ok with failing often. 

Now that we can agree that innovation is messy and it feels wasteful, what can we do about it? 

1. Don’t start with a big spectacular thing. Instead, try to come up with an MVP (minimum viable product) that you can build on (or next to) in the future. 

2. Budget for the messiness and the learning process. Make sure you have enough money to make the mistakes required to get the learning experience you need to bring your idea to life.

3. Aim for many small mistakes, so you don’t make one massive “end of game” mistake. This idea expands on (1) above. Move fast, but take small steps. This approach will make it easier to backtrack and change direction. Significant commitments are giant leaps forward that give you less flexibility to turn around. 

4. Don’t worry about optimization and edge cases in the beginning. If you do, you may end up doing tedious and lengthy work on a feature that may not even make it into the final product.

5. Try again tomorrow. Some days it may feel like you are getting nowhere, and this is all doomed to fail. That is normal. Take a break, go back to the original vision that got you excited and try again tomorrow. 

6. Be patient. You are playing the long game.

7. Once you have your MVP, you can start again and “do it right” this time. It will no longer be innovation because you have learned your lessons. Now is the time for the polished, optimized, and secured product. 

How do you deal with innovation in your projects?

The self-diagnosed client

The self-diagnosed client is the person who comes to you with a problem, a solution, and they only need you to implement that solution for them!

Does that seem right to you?

Consider the following scenario. A person goes to the doctor; this person knows the problem; knows the treatment and only asks the doctor for the medicine they want. In most places, it would be a case of mal praxis for the doctor to accept the self-diagnosis.

At the very least, the doctor would ask questions to confirm what the real problem is and confirm the diagnosis, right?

When it comes to online businesses, the issue is not that clear cut, and it depends on what you are selling.

If what you are offering is creative, custom made solutions, then a self-diagnosed client is a disaster waiting to happen. You cannot know if the two of you are a good fit if you have not followed your own diagnosis process that will enable to serve the client best. The client doesn’t know what they don’t know… they have a blind spot. If they knew everything, they would not come to you for a custom made solution. So it is your job to at least confirm the problem and what they think the best solution might be :).

In case of a more standardized offer, then it is indeed up to the client to make up their mind if your offer is a good fit for what they want to do. You can, of course, help make up their mind with excellent communication about your product or service.

To Backup or not to Backup

Some years ago I had the opportunity to work alongside a veteran software developer. That was a treat for me and also a way to learn big lessons fast.

I remember being overconfident in my abilities, fresh out of school, and making silly mistakes when all that knowledge had to be put into practice.

I wanted to be quick, and agile, and free! I wanted to get in, fix the problem and move on!

But there was an incident that taught me a valuable lesson.

The server we were managing got hacked and crashed.

Working alongside the Veteran we managed to identify the security vulnerability, fix it and then restore the website within 6 hours. This was a big and popular forum. 6 hours recovery time was much shorter than the couple of days that this usually takes.

Shortly after restoring access, I heard from one of the members saying: “The way you recovered from this and the speed at which you did it is nothing short of impressive. In my career, I have worked for big software companies and none of them have in place such a good recovery plan.”

I could not take much credit for that, so I decided to pay attention to “the Old Veteran” because it was clear now he knew was he was doing :).

The Importance of Backups

We were able to bounce back so quickly because we had backups. Now only that, but we had versioned backups. Meaning we could go “back in time” to before the problem, see what changed and fix it. And then restore almost all of the user data, with minimal loss. Without versioned backups, this process would have been long and tedious and I do not know if we would have been able to spot the point of entry.

This is a happy ending story and here is what I have learned:

1. You always do backups – even if you think you don’t need them.

2. You test your backups – an untested backup is no backup. I have a story here where a client was paying their hosting company for a remote backup system and when the time came to use it, the backups were corrupted and so not usable.

3. You never delete things – you rename them and then archive them – this way you can always retrace your steps back to something that was working

4. When writing software you always, always use source control – which is basically a system that does smart backups of your work that allow you to “go back in time” and fix problems.

A beginner’s mistake- “I am too good for Backups”

As I have said, fresh out of school, I had bright ideas and I wanted to move very fast, but I did not ever have to deliver work that was used by real people, in a real situation, facing potential attacks from real online threats.

When you are prototyping and testing out an idea, it is OK to be quick, because if the idea is bad or not useful, you need to find out fast. But once you have something that you want to build out for the long term, then you need to switch gears and sacrifice reaction speed for being more organized.

I confess that this did not make sense to me for a long time. But as I worked in bigger and bigger projects it became obvious how the “slow work” of thinking of a structure to organize your code, setting up source control and doing backups was actually the fast lane. Why? Because it reduces risk and allows you to easily maintain the project as you move forward.

The opposite of this is working at neck-breaking speed, not “wasting time” with backups or source control, in order to put something on the market quickly. All the projects that I managed or I was a part of, that did not put in the time to be organized, eventually ground to a halt and had to be abandoned or rewritten.

I have done this mistake enough times to learn my lesson: for quality and sustainable work always do backups and use source control.

Client’s point of view – Do backups make business sense?

It is now obvious for me that backups are not just a good idea. But why should you care about them?

It depends on how well you can manage risk and how important is your data and your customers’ data to you and your business.

If you can afford to lose it all, then you don’t need backups.

If you can afford the downtime of having to rebuild your application from scratch, then you don’t need backups.

But in my opinion, good backups are a cost-effective way to mitigate the security and data loss risks associated with running an online business.

Do you have a backup policy in place? And if you do, have you tested your backups lately to make sure that you will find in there what you expect to find?