Empirical Study on Refactoring Large-Scale Industrial Systems and Its Effects on Maintainability
Gábor Szőke, Gábor Antal, Csaba Nagy, Rudolf Ferenc, Tibor Gyimóthy
Department of Software Engineering, University of Szeged, Hungary
Abstract
Software evolves continuously, it gets modified, enhanced, and new requirements always arise. If we do not spend time occasionally on improving our source code, its maintainability will inevitably decrease. The literature tells us that we can improve the maintainability of a software system by regularly refactoring it. But does refactoring really increase software maintainability? Can it happen that refactoring decreases the maintainability? Empirical studies show contradicting answers to these questions and there have been only a few studies which were performed in a large-scale, industrial context. In our paper, we assess these questions in an in vivo context, where we analyzed the source code and measured the maintainability of 6 large-scale, proprietary software systems in their manual refactoring phase. We analyzed 2.5 million lines of code and studied the effects on maintainability of 315 refactoring commits which fixed 1,273 coding issues. We found that one single refactoring made only a very little difference (sometimes even decreases maintainability), but a whole refactoring period, in general, can significantly increase maintainability, which can result not only in the local, but also in the global improvement of the code.
Overview
In the research project, company programmers were required to refactor their own code, hence improve its maintainability, but they were free to choose how they wanted to do it. They could freely choose any coding issues or metrics from the reported problems, and they were also free to identify additional problems in the code. We selected 6 of these systems and for each system, we analyzed the maintainability of the revisions where developers committed refactorings and the revisions before these commits.
Details are presented in our paper, submitted to the special issue of Journal of Systems and Software dedicated to IEEE SCAM 2014 conference papers (14th International Working Conference on Source Code Analysis & Manipulation).
Results
Developers preferred to fix concrete coding issues rather than fix code smells
We found that when developers had the extra time and budget to refactor their code, they optimized their process so as to improve the maintainability of a system. Our results suggest that developers did not really want to improve the metric values or avoid certain antipatterns in their code; they simply went for the concrete problems and fixed coding issues. In our study we examine more than a thousand coding issue fixes and their effect on maintainability.
A single refactoring had only a small impact (sometimes even negative effect)
We applied the ColumbusQM maintainability model to measure changes in the maintainability of large-scale industrial systems before/after refactoring commits. Our measurements revealed that the maintainability changes induced by refactoring operations can be seen in most of the cases. One particular change usually caused only a small change, hence some refactorings (mostly those involving fixing local coding issues) the model did not display any changes in the maintainability.
Measurements reveal that some refactoring operations might have a negative impact on the maintainability of the system, although its main purpose is to improve it. It is not easy to decide how to fix an issue and balance its effects as it might happen that we want to improve one maintainability attribute, but we debase others.
Whole refactoring process has significant beneficial effect on the maintainability
After the refactoring period, the overall maintainability of the software systems improved and the maintainability model was able to measure this improvement in five out of the six systems. Commits which fixed more coding issues had a relatively higher impact on maintainability. Similarly, we observed in the tables that when developers fixed more metrics or antipatterns together, they induced a bigger change compared to others. Hence, a larger refactoring has a noticeable, positive impact on the maintainability, which is measurable using static analysis techniques.
The reason for this is not only because we improve the maintainability of our software, but also because developers will learn from the process and pay more attention to writing better maintainable code.
| System | Metrics | Anti-patterns | Coding Issues | Maintain. Before | Maintain. After | Total Impr. | Refactoring Impr. |
|---|---|---|---|---|---|---|---|
| System A | 0 | 0 | 469 | 5.4699 | 5.3193 | -0.1506 | -0.0030 |
| System B | 36 | 41 | 204 | 5.8095 | 5.8762 | 0.0667 | 0.0135 |
| System C | 9 | 8 | 586 | 3.4629 | 3.7354 | 0.2725 | 0.0767 |
| System D | 6 | 0 | 19 | 5.4775 | 5.6594 | 0.1819 | 0.0151 |
| System E | 17 | 11 | 35 | 6.4362 | 6.8190 | 0.3828 | 0.0436 |
| System F | 22 | 18 | 50 | 6.4972 | 6.5926 | 0.0954 | 0.0716 |
Download
This appendix contains the following additional data:- ap-antipatterns.txt : The list of antipatterns we showed the developers on the start of the project.
- ap-metrics.txt : The list of metrics we showed the developers on the start of the project.
- ap-pmds.txt : The list of coding rule violations (coding issues) we showed the developers on the start of the project.
- SystemA-1.csv : Raw data of the analyzed project.
- SystemA-2.csv : Raw data of the analyzed project.
- SystemA-3.csv : Raw data of the analyzed project.
- SystemA-4.csv : Raw data of the analyzed project.
- SystemB.csv : Raw data of the analyzed project.
- SystemC.csv : Raw data of the analyzed project.
- SystemD.csv : Raw data of the analyzed project.
- SystemE.csv : Raw data of the analyzed project.
- SystemF.csv : Raw data of the analyzed project.