Performance improvements… or experiments?

The project I’m involved in will face great challenges in the upcoming years. That was the reason why we have decided to focus on improving performance of the application before these improvements become a necessity.
How to increase efficiency? Stability? Responsiveness? How to prepare for future? That was the question that we asked ourselves and our goal was to find the answers.

We prepared the list of the challenges and potential improvements that can help us face them. Then we had to choose the most important items from the list. Each of us had a chance to share their opinion, share their view and suggestions.
During this conversation one of us reminded us about the most important activity that needs to take place before any other - we have to improve the measurement of our application’s performance.

We cannot start to improve, before we’ve got nothing but assumptions!

Isn’t it obvious?

I believe you know you shouldn’t start to change your code if there are no tests to prove that the change didn’t break anything? It is obvious nowadays, isn’t it?

What about performance? Should we improve performance with only good feelings, assumptions and arguments as a justification and proofs?
The answer is simple, we should not. However, we often treat performance as something that is ok to play with. To mess up with. To experiment with.

The question is why?

It’s not so easy to measure

If we don’t check what’s the performance of our application on given conditions right now, how we can be sure that our changes will improve it. Because we assume so? Quite doubtful.

People are lazy. Most of us chose easier and faster solutions if it is possible. It is faster NOT to spend your time looking at the performance. Just go for the improvements!
And measuring the performance is not only about time and effort.

It’s relatively simple to find what scenarios we need to test when we are talking about the code. We expect that after each invocation of a particular method we will observe the same effect. What about performance?
Of course, we should check whether our application behaves as expected with specific load, but this is not always what we need. Not in case when our application does not fulfill the expectations. To start work on improvements, we need to find a problem. To find a problem, we need to have information. To have this information we need to monitor various places in our application. This is not only about data from today. It’s about historical reports as well.
We need to know what and where was changed. We need to have enough information to start analysis.

There are many things that have to be done. It’s not only about tests that confirm performance is good today. It’s not only about how much time it takes to process the requests. We need to gather (and store!) information about many points in our application. Information about time, load and resources’ consumption.
And it won’t take care of itself. Someone needs to keep an eye on all infrastructure.

What should we measure?

You have to find places in your application which give you the most useful information. You can measure everything, but, first of all, who will pay for the time and effort? Before you are able to monitor everything, you will have spent tremendous amount of time for research, setting up everything, etc.
And then data analysis comes. If we assume you will gather any possible data who will analyze them? The more data, the more information. But the more data, the less chances for spotting the correlations between statistics. Also such analysis will cost you more time and effort.

That’s why you have to ask yourselves what kind of data will give you the most valuable information.

Where’s the fun?

When you know what needs to be improved, you will be more eager to start. It is fun. Making the application better, maybe doing some experiments, learning. It makes you happy.
But before you can actually start with those activities, you need to find those places that require improvements.
And this is like writing tests. We know that is valuable, but… well… where’s the fun?

Business doesn’t need it

You are paid for your work. People who pay you wants to see results. For them measuring the performance of the application is yet another activity that requires time and people and especially money. And brings nothing in return.

That’s why it is so important to focus on the most crucial parts of the application.
You have to minimise costs, but without losing too much value.

But…

Even if we have to face all of those challenges, we are professionals. We are the value-driven people and even if an activity is not fun at all. Even when it is not so easy we are looking at the given values and this is what pushes as forward.

The sooner you will start, the better. The sooner you will start, the less effort it will cost.

Your application grows. You cannot stop it. You probably even wouldn’t like to stop it. However, with growth there are more places that will require your attention.
Don’t wait until the amount will overwhelm you. Identify the crucial places and start.

Don’t wait till the day when “user measurement” will tell you that something is wrong.

You may like it or not, but it has to be obvious that before any change you have to measure how the thing that you want to improve works.