When to rewrite the project code and how to explain this to the customer

In this article, we will consider a situation where programmers say: “We need to rewrite everything,” but it’s hard to convey it to the customer.

We will figure out what to do with this and how to reconcile programmers and business, if you are a Lead of a project, Project Manager or programmer who wants improvements, but they seem to not hear you. Most likely, you know firsthand about this problem: this happens either with you, then with your friends or colleagues, and once again the discussion begins: what to do?

Whatever your final decision – not to rewrite, rewrite or partially do it – I hope this detailed analysis will help to make it wisely and deliberately, as well as convey your idea to other stakeholders.

Project stakeholders

The more interested people in the project, the more diverse their goals. Let’s identify all the stakeholders involved in the project. Note that you may not even be familiar with some of them. But they are, and they have an influence on certain decisions.

    1.Investors – because they invest their financial resources in this project and expect a good Return on Investment (ROI).

    2.Product Owner is the person who manages this project and prioritizes the functionality that you are developing. He communicates with programmers and investors.

    3.Users are the ones who use your product and most likely pay for it.

    4.Programmers – yes, they are also stakeholders, the project should suit them, otherwise you will not have programmers.

Sorry, but that’s all! The magic that happens inside is interesting only to programmers. While the program does what it needs to, it suits the owner of the product, investors, and even users.

Why programmers want to rewrite project code

Firstly, because we have a need to do well. Unfortunately, the concept of “good” is different for everyone. For example, I would like to rewrite the code when I got a legacy project, and those who wrote it had a different concept, what is “good”, different from yours. Or if you got a prototype, which was done in a hurry, and the programmers themselves realized that they did not work very well.

Secondly, because during operation we load information into memory. And it’s much easier for us to keep structured code in our heads than a mess of files and functions.

Thirdly, for work we need to understand what we are working with. When you change what you don’t understand, it causes stress. Who wants to work in constant stress during development, testing, deployment?

Where the interests of programmers and other stakeholders overlap

In addition to functional characteristics, programs have non-functional characteristics. These include:

•     response time (performance);
•     lack of bugs (reliability, usability);
•     availability
•     resistance to failures (resilience);
•     extendability
•     maintainability
•     average repair time;
•     cost of development and support (cost
•     testability);
•     auditability and debugging.
  

The first four characteristics are of great interest to users, they are visible to everyone. More precisely, these characteristics do not interest users until they become bad.

The rest of the characteristics – starting with extensibility – are of interest to the product owner and programmers, but they are necessary to maintain the first four at the proper level. And also in time to release new features.

So, it is in the zone of non-functional characteristics that the point of intersection of the interests of programmers, product-owner and users is located. When asking for refactoring, indicate which non-functional characteristics you are currently not comfortable with.

If the project has a very complex and confusing code, then, most likely, maintainability, development speed and testability are weak. If there are old libraries and frameworks, then most likely you have problems with extensibility. If the architecture did not anticipate a large number of requests, but they are expected, these are future problems with scalability and performance. If you spend half of your work time searching for the causes of bugs, it’s clearly worth improving auditability & traceability.

Your arguments should be based on what problems the project currently has or what problems will appear in the near future if the situation is not changed (preventive risk management).

Why the parties cannot agree

At one of the trainings, we were taught in any conflict situation to write out the desires and concerns of the parties. The technique is good, and I’ll just leave a table here that you can fill out based on your situation:

Conflict most often occurs where the desires of one side intersect with the fears of the other. Because of this, the parties cannot agree. What some stakeholders want is a concern for others.

Note: if an outsourcing company is still involved in this, then fears of investors are added to the fear that they simply want to sell additional man-hours of development. I see no other way to dispel this fear, except to order an audit of the project from a third disinterested party trusted by Product Owner.

Experts say

Joel Spolsky wrote 20 years ago:

    “The fundamental law of programming is that code is harder to read than to write. Any code already written seems too complicated for the programmer, so he wants to write it again. Writing your own functions is easier and more fun than understanding existing code”.

The problem is that when the next programmer arrives, he will again find the code complicated, and will want to rewrite it again. To prove this axiom [that code is easier to write than to read], ask almost any programmer about the code that he is currently working with. He will answer that this is still horrible.

    “Nobody wants to do those parts of the work that are not fun. Repairing bugs is not fun. Writing from scratch is fun, ”writes Jamie Zawinski.

So do not rewrite it at all?

Invalid output. The principle of “if works – do not touch” is also not suitable. The book “Object-Oriented Reengineering Patterns” mentions that if the system is functioning, but you cannot support or expand it, it is broken.

Basecamp released three versions of the product: Basecamp, Basecamp 2, and Basecamp 3. They did not destroy or globally modify the previous version of the product, but simply released a new one.

Roughly the same thing happened with AngularJS – when there was a desire to rewrite it, a new Angular framework was released, and thus the risks of breaking what already works were mitigated.

Microsoft rewrote Visual Studio and released VSCode, which many, I think, use. Google rewrote its Inbox, but left both versions working – the old Inbox and the new GMail.

Good reasons to rewrite a project

•     You cannot add something new without rewriting the old.
•     It is very difficult to introduce new people to the project, it takes more than two months.
•     Unable to configure Continuous Integration or deploy.
•     Simple bug fixes take a lot of time.
•     The platform on which the application runs is no longer supported.
•     An increase in the number of users is expected, which the old system will not stand.
•     The interface is obsolete, and you are rewriting it to a more modern one.
  

Unworthy reasons to rewrite a project

•     It is written on promises, and I want async / await.
•     Here is the N framework, and my friends say, now fashionable Y.
•     I want to add a couple of technologies to my resume.
•     I do not like code that I did not write.
•     The code works, but I don’t understand how.

Pitfalls

•  Rewriting always takes longer than expected (according to world statistics mentioned in McConnell’s book “How much does a software product cost”, software projects are usually underestimated by 30% or more).
•     Simple rewriting gives little value to the end user.
•     You will “forget” or “lose” part of the functionality if there is no complete documentation for the old system.
•     You will make the same bugs that previous programmers have already done and repaired before you.
•     When rewriting from scratch, you will have to support both versions, old and new, for some time.

Rewrite or refactor?

Overwriting (rewrite from scratch) is when you re-write code from scratch using the old read-only code. Refactoring is when you come to a new look through successive transformations of the old code.

An interesting question to which I will not give an answer, but maybe you will give in the comments: is it possible to come to the same result by refactoring instead of rewriting? As far as I understand, it’s possible to simply rewrite it sometimes — faster than refactoring. Or it seems faster.

But refactoring is considered safer for the following reasons:

•     it is done in small steps, and small changes are easier to plan, test and release;
•     if for some reason the rewriting is stopped halfway, the customer will not be in a situation where the new version of the product has not yet been added, and the old one remains the same as it was.

And where is your plan!

Did you make repairs? Remember what interests you? Price, quality and terms, right? It’s the same here. In order for the customer to say yes to you, provide him with information about the price, quality and terms.

Price and Dates

Price and timing are closely related. In order to make the most accurate estimate, you need a work plan. In order to make a work plan – you need a clear understanding of what you want to get as a result:

•     Make diagrams of what you want to get as a result.
•     Explain in the comments or in the design document why the new structure is better than the old.
•     Break the work into stages and make an estimate (optimistic, pessimistic, realistic).
•     Think over risks (for example, leaving the key developer).
•     Who and how will test the result, do you have a checklist for full testing?
•     Do not forget to include in the estimate work on setting up deployment, data transfer and user migration.
  

With this plan you can appeal to stakeholders with a proposal to rewrite.

Quality

Quality is most often guaranteed by your reputation. If you are the very guys who wrote poorly, where is the guarantee that this time you will do well? In order for you to be allowed to do such risky things with the project, you need to have a high degree of trust from stakeholders. If they do not give good, it means that the customer has not been informed about the value, he considers the price too high or does not trust. Ce la vie.

A few words about engineering ethics

Upon entering the profession, doctors take the Hippocratic oath. There are several points in it, and one of them – “the dominant of the patient’s interests” – in the process of treatment, the doctor undertakes to follow the interests of the patient, and not his own. Whether he will actually do this is a matter of the doctor’s education.

Upon entering the profession of a programmer, we do not [yet] take any oaths, but a code of ethics for the programmer exists here and here, and a similar clause is also there: “Principle 2: Software engineers shall act in a manner that is in the best interests of their client and employer consistent with the public interest” 

To pursue your personal goals, not business goals, is unethical.

Give the numbers

There are people who make decisions based on numbers, not abstract stories, how everything will be fine. If your Product Owner is just like that, provide calculations where you compare the cost of rewriting, the cost of supporting a new version of a product, and the cost of supporting an old version of a product.

The cost of rewriting is easiest to calculate, it is directly proportional to the time it takes to rewrite and roll out the new version.

The cost of product support consists of the following components: this is the time spent on developing new features, bug fixes and testing, introducing a person to the project, deployment, as well as the minimum level of programmers necessary for support and understanding (Senior vs Junior).

If you reduce these indicators, it means that you have reduced the cost of support. Also, rewriting pays off in the event that an increase in the number of users is expected, which the old system simply can not stand.

Choose the right time

If you have made good arguments and developed a detailed plan, Product Owner may still have a good reason to refuse: “Not now”.

Indeed, the roadmap of the project is being developed for months to come, and suddenly big changes cannot be added to the plan. Also, you may not be aware of the financial condition of the project – is it “breathing in the air and trying to survive” or “received the third round of investments, and development is planned”? It is easier to get time and resources for refactoring in the second case.

You can discuss with Product Owner when, in his opinion, all this can be realized, and be patient. Think: is it possible to rewrite not all, but part? Identify the most problematic parts of the project by looking at the history of bug reports or user complaints.

Summary

If it seems to you that your project needs to be rewritten:

•     Make sure the reasons are good, and not just “I don’t like this code”.
•     Find the intersection point of your interests and those of other stakeholders.
•     Identify stakeholder concerns and how to reduce them.
•     Make a detailed description of what you want to get as a result: schemes, diagrams, design document.
•     Think about why you cannot come to the same result by refactoring.
•     Make an optimistic, realistic and pessimistic estimate.
•     Compare the costs and benefits resulting from it.
•     Choose the right time.
•     Enlist the trust of decision makers.
•     Try to change in stages and start with the most critical parts.