Effort Estimation in Agile Software Development: A Systematic Map Study

Authors

  • Camilo Andrés Piñeros Rodríguez Universidad del Cauca - Research group GTI. Popayán, (Colombia)
  • Luz Marina Sierra Martinez Universidad del Cauca - Research group GTI. Popayán, (Colombia)
  • Diego Hernán Peluffo Ordoñez Universidad Mohammed VI Polytechnic Ben Guerir, (Marruecos)
  • Jimena Adriana Timana Peña Universidad del Cauca - Research group GTI. Popayán, (Colombia)

DOI:

https://doi.org/10.17981/ingecuc.19.1.2023.03

Keywords:

Effort Estimation, Agile Software Development, Issues and Challenges, Automatic Learning, Performance Metrics

Abstract

Introduction − Making effort estimation as accurate and suitable for software development projects becomes a fundamental stage to favor its success, which is a difficult task, since the application of these techniques in constant changing agile development projects raises the need to evaluate different methods frequently. 

Objectives− The objective of this study is to provide a state of the art on techniques of effort estimation in agile software development (ASD), performance evaluation and the drawbacks that arise in its application. 

Method− A systematic mapping was developed involving the creation of research questions to provide a layout of this study, analysis of related words for the implementation of a search query to obtain related studies, application of exclusion, inclusion, and quality criteria to filter nonrelated studies and finally the organization and extraction of the necessary information from each study.  

Results− 25 studies were selected; the main findings are: the most applied estimation techniques in agile contexts are: Estimation of Story Points (SP) followed by Planning Poker (PP) and Expert Judgment (EJ). The most frequent solutions supported in computational techniques such as: Naive Bayes, Regression Algorithms and Hybrid System; also, the performance evaluation measures Mean Magnitude of Relative Error (MMRE), Prediction Assessment (PRED) and Mean Absolute Error (MAE) have been found to be the most commonly used. Additionally, parameters such as feasibility, experience, and the delivery of expert knowledge, as well as the constant particularity and lack of data in the process of creating models to be applied to a limited number of environments are the challenges that arise the most when estimating software in agile software development (ASD)   

Conclusions− It has been found there is an increase in the number of articles that address effort estimation in agile development, however, it becomes evident the need to improve the accuracy of the estimation by using estimation  techniques supported in machine learning  that have been shown to facilitate and improve the performance of this. 

Downloads

Download data is not yet available.

References

E. Mendes, Cost estimation techniques for web projects. HYS, PA: IGI Pub, 2007. https://doi.org/10.4018/978-1-59904-135-3

M. Ramessur & S. Nagowah, “A predictive model to estimate effort in a sprint using machine learning techniques,” Int J Comput Sci Inf Technol, vol. 13, no. 7, pp. 1101–1110, Apr. 2021. https://doi.org/10.1007/s41870-021-00669-z

R. Britto, E. Mendes & J. Borstler, “An Empirical Investigation on Effort Estimation in Agile Global Software Development,” presented at 10th International Conference on Global Software Engineering Workshops, ICGSEW, CR, ES, 13-16 Jul. 2015. https://doi.org/10.1109/ICGSE.2015.10

S. Bilgaiyan, S. Mishra & M. Das, “A Review of Software Cost Estimation in Agile Software Development Using SoftComputing Techniques,” presented at International Conference on Computational Intelligence and Networks, CINE, BBSR, IN, 11-11 Jan. 2016. https://doi.org/10.1109/CINE.2016.27

IEOM, Annual IEEE Computer Conference, International Conferenceon Industrial Engineering and Operations Management, IEOM, DXB, UAE, 3-5 March 2015. Available: https://ieomsociety.org/ieom/

S. Rc, M. Sánchez-Gordón, R. Colomo-Palacios & M. Kristiansen, “Effort Estimation in Agile Software Development: AExploratory Study of Practitioners’ Perspective,” in LASD 2022: Lean and Agile Software Development, Przybyłek, A.,Jarzębowicz, A., Luković, I., Ng, Y. (Eds)., Cham, CH: Springer, 2022, vol. 428, pp. 136–149. https://doi.org/10.1007/978-3-030-94238-0_8

H. Rastogi, S. Dhankhar & M. Kakkar, “A Survey on Software Effort Estimation Techniques,” presented at 5th International Conference - Confluence The Next Generation Information Technology Summit, Confluence, NOI, IN, 25-26 Sep. 2014. https://doi.org/10.1109/CONFLUENCE.2014.6949367

P. Salvetto, “Modelos automatizables de estimación muy temprana del tiempo y esfuerzo de desarrollo de sistemas de información,” Tesis doctoral, Fac Inform, UPM, MAD, ES, 2004. Recuperado de https://oa.upm.es/367/1/PEDRO_SALVETTO_LEON.pdf

E. Dantas, M. Perkusich, E. Dilorenzo, D. Santos, H. Almeida & A. Perkusich, “Effort Estimation in Agile Software Development: An Updated Review,” Int J Softw Eng Knowl Eng, vol. 28, no. 11–12, pp. 1811–1831, Nov. 2018. https://doi.org/10.1142/S0218194018400302

B. Alsaadi & K. Saeedi, “Data-driven effort estimation techniques of agile user stories: a systematic literature review,” Artif Intell Rev, vol. 55, no. 7, pp. 5485–5516, Jan. 2022. https://doi.org/10.1007/s10462-021-10132-x

K. Petersen, S. Vakkalanka & L. Kuzniarz, “Guidelines for conducting systematic mapping studies in software engineering: An update,” Inf Softw Technol, vol. 64, pp. 1–18, Aug. 2015. https://doi.org/10.1016/j.infsof.2015.03.007

M. Fernández-Diego, E. Méndez, F. González-Ladrón-De-Guevara, S. Abrahão & E. Insfran, “An update on effort estimation in agile software development: A systematic literature review,” IEEE Access, vol. 8, pp. 166768–166800, Sep. 2020. https://doi.org/10.1109/ACCESS.2020.3021664

M. Usman, E. Mendes, F. Weidt, & R. Britto, “Effort estimation in Agile Software Development: A systematic literature review,” presented at 10th International Conference on Predictive Models in Software Engineering, PROMISE '14, TO, IT, 17 sep. 2014. https://doi.org/10.1145/2639490.2639503

T. Hacaloglu & O. Demirors, “Challenges of Using Software Size in Agile Software Development: A Systematic Literature Review,” presented at the Academic Papers at IWSM Mensura, IWSM-Mensura, BJ, CN, 19-20 Sep. 2018. Available: https://hdl.handle.net/11147/7045

A. Altaleb & A. Gravell, “Effort Estimation across Mobile App Platforms using Agile Processes: A Systematic Literature Review,” JSW, vol. 13, no. 4, pp. 242–259, Apr. 2018. https://doi.org/10.17706/jsw.13.4.242-259

B. Kitchenham & S. Charters, “Guidelines for Performing Systematic Literature Reviews in Software Engineering Version 2.3,” KUSU and UoD, Staf, UK, EBSE 2007-001 Tech Rep, 2007. Available from https://userpages.uni-koblenz.de/~laemmel/esecourse/slides/slr.pdf

B. Kitchenman & D. Budgen, Evidence-Based Software Engineering and Systematic Reviews. BC RTN, FL, USA: CRC Press Taylor & Francis Group, 2015.

K. Felizardo, E. Mendes, M. Kalinowski, E. Souza & N. Vijaykumar, “Using Forward Snowballing to update Systematic Reviews in Software Engineering,” presented at 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM '16, BC RTN, FL, USA, 8-9 Sep. 2016. https://doi.org/10.1145/2961111.2962630

B. Kitchenman, O. Brereton, D. Budgen, M. Turner, J. Bailey & S. Linkman, “Systematic literature reviews in software engineering - A systematic literature review,” Inf Softw Technol, vol. 51, no. 1, pp. 7–15, Jan. 2009. https://doi.org/10.1016/j.infsof.2008.09.009

F. Yaghmalef, “Content validity and its estimation,” JME, vol. 3, no. 1, pp. 25–27, Mar. 2003. Available: https://brieflands.com/articles/jme-105015.pdf

E. Almanasreh, R. Moles & T. Chen, “Evaluation of methods used for estimating content validity,” Res Social Adm Pharm, vol. 15, no. 2, pp. 214–221, Feb. 2019. https://doi.org/10.1016/j.sapharm.2018.03.066

E. Milian, M. de Spinola & M. de Carvalho, “Fintechs: A literature review and research agenda,” Electron Commer Res Appl, vol. 34, Feb. 2019. https://doi.org/10.1016/j.elerap.2019.100833

M. Hamid, F. Zeshan, A. Ahmad, F. Ahmad, M. Hamza, Z. Khan, S. Munawar & H. Aljuaid, “An Intelligent Recommender and Decision Support System (IRDSS) for Effective Management of Software Projects,” IEEE Access, vol. 8, pp. 140752–140766, Jul. 2020. https://doi.org/10.1109/ACCESS.2020.3010968

M. Choetkiertikul, H. Dam, T. Tran, T. Pham, A. Ghose & T. Menzies, “A Deep Learning Model for Estimating Story Points,” ITSE, vol. 45, no. 7, pp. 637–656, Jan. 2018. https://doi.org/10.1109/TSE.2018.2792473

A. Kaushik, D. Tayal & K. Yadav, “A Comparative Analysis on Effort Estimation for Agile and Non-agile Software Projects Using DBN-ALO,” Arab J Sci Eng, vol. 45, no. 4, pp. 2605–2618, Nov. 2019. https://doi.org/10.1007/s13369-019-04250-6

O. Malgonde & K. Chari, “An ensemble-based model for predicting agile software development effort,” Empir Softw Eng, vol. 24, no. 2, pp. 1017–1055, Apr. 2019. https://doi.org/10.1007/s10664-018-9647-0

S. Bilgaiyan, S. Mishra & M. Das, “Effort estimation in agile software development using experimental validation of neural network models,” Int J Inf Technol, vol. 11, no. 3, pp. 569–573, Abr. 2018. https://doi.org/10.1007/s41870-018-0131-2

S. Butt, S. Misra, J. Diaz-Martinez & F. De la Hoz, “Efficient Approaches to Agile Cost Estimation in Software Industries: A Project-Based Case Study,” presented at Information and Communication Technology and Applications, ICTA 2020, Cham, CH, 24-27 Nov. 2021. https://doi.org/10.1007/978-3-030-69143-1_49

W. Alsaqaf, M. Daneva & R. Wieringa, “Quality requirements challenges in the context of large-scale distributed agile: An empirical study,” Inf Softw Technol, vol. 110, pp. 39–55, Mar. 2018. https://doi.org/10.1016/j.infsof.2019.01.009

M. Gultekin & O. Kalipsiz, “Story Point-Based Effort Estimation Model with Machine Learning Techniques,” IJSEKE, vol. 30, no. 1, pp. 43–66, Jan. 2020. https://doi.org/10.1142/S0218194020500035

M. Alhamed & T. Storer, “Playing Planning Poker in Crowds: Human Computation of Software Effort Estimates,” presented at 43 International Conference on Software Engineering, ICSE, MAD, ES, 22-30 May. 2021. https://doi.org/10.1109/ICSE43902.2021.00014

M. Arora, A. Sharma, S. Katoch, M. Malviya & S. Chopra, “A State of the Art Regressor Model’s comparison for Effort Estimation of Agile software,” presented at 2nd International Conference on Intelligent Engineering and Management, ICIEM, LDN, UK, 28-30 Apr. 2021. https://doi.org/10.1109/ICIEM51511.2021.9445345

A. Sharma & N. Chaudhary, “Linear Regression Model for Agile Software Development Effort Estimation,” presented at 5th IEEE International Conference on Recent Advances and Innovations in Engineering, ICRAIE, JAIP, IN, 1-3 Dec. 2020. https://doi.org/10.1109/ICRAIE51050.2020.9358309

P. Sudarmaningtyas & R. Mohamed, “Extended Planning Poker: A Proposed Model,” presented at 7th Inter­national Conference on Information Technology, Computer, and Electrical Engineering, ICITACEE, SRG, ID, 24-25 Sep. 2020. https://doi.org/10.1109/ICITACEE50144.2020.9239165

J. Angara, S. Prasad & G. Sridevi, “DevOPs project management tools for sprint planning, estimation and execution maturity,” Cybern Inf Technol, vol. 20, no. 2, pp. 79–92, Mar 2020. https://doi.org/10.2478/cait-2020-0018

H. Sheemar & G. Kour, “Enhancing User-Stories Prioritization Process in Agile Environment,” presented at International Conference on Innovations in Control, Communication and Information Systems, ICICCI, GRT NOI, IN, 12-13 Aug. 2017. https://doi.org/10.1109/ICICCIS.2017.8660760

L. Radu, “Effort prediction in agile software development with Bayesian networks,” presented at 14th Inter­national Conference on Software Technologies, ICSOFT, STBL, PT, 26-28 Jul. 2019. https://doi.org/10.5220/0007842802380245

E. Dantas, A. Costa, M. Vinicius, M. Perkusich, H. Almeida & A. Perkusich, “An effort estimation supporttool for agile software development: An empirical evaluation,” presented at 31th International Conference on SoftwareEngineering and Knowledge Engineering, SEKE, LX, PT, 10-12 Jul. 2019. https://doi.org/10.18293/SEKE2019-141

H. Premalatha & C. Srikrishna, “Effort estimation in agile software development using evolutionary cost- sensitive deep Belief Network,” Int J Intell Eng Syst, vol. 12, no. 2, pp. 261–269, Dec. 2018. https://doi.org/10.22266/IJIES2019.0430.25

T. Khuat & M. Le, “A Novel Hybrid ABC-PSO Algorithm for Effort Estimation of Software Projects UsingAgile Methodologies,” JISYST, vol. 27, no. 3, pp. 489–506, Mar. 2017. https://doi.org/10.1515/jisys-2016-0294

E. Scott & D. Pfahl, “Using developers’ features to estimate story points,” presented at InternationalConference on the Software and Systems Process, ICSSP'18, GBG, SE, 26-27 May. 2018. https://doi.org/10.1145/3202710.3203160

P. Ram, P. Rodriguez & M. Oivo, “Software Process Measurement and Related Challenges in Agile SoftwareDevelopment: A Multiple Case Study,” presented at Intetnational Conference Product-Focused Software Process Improvement, PROFES, WOB, DE, 28-30 Nov. 2018. https://doi.org/10.1007/978-3-030-03673-7_20

C. Prasada Rao, P. Siva Kumar, S. Rama Sree & J. Devi, “An agile effort estimation based on story points usingmachine learning techniques,” presented at 2nd International Conference on Computational Intelligence and Informatics, ICAI, HYD, IN, 22-23 Dec. 2018. https://doi.org/10.1007/978-981-10-8228-3_20

A. Kialbekov, “Empirical Study on Commonly Used Combinations of Estimation Techniques in Software Development Planning,” presented at European Symposium on Software Engineering, ESSE '20, ROM, IT, 6-8 Nov. 2020. https://doi.org/10.1145/3393822.3432328

A. Altaleb and A. Gravell, “An Empirical Investigation of Effort Estimation in Mobile Apps Using Agile Development Process,” JSW, vol. 14, no. 8, pp. 356–369, Jul. 2019. https://doi.org/10.17706/jsw.14.8.356-369

Downloads

Published

2023-01-29

How to Cite

Piñeros Rodríguez, C. A., Sierra Martinez, L. M., Peluffo Ordoñez, D. H., & Timana Peña, J. A. (2023). Effort Estimation in Agile Software Development: A Systematic Map Study. INGE CUC, 19(1), 22–36. https://doi.org/10.17981/ingecuc.19.1.2023.03

Issue

Vol. 19 No. 1 (2023): (Enero - Junio)

Section

In Press

License

Copyright (c) 2022 INGE CUC

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Published papers are the exclusive responsibility of their authors and do not necessary reflect the opinions of the editorial committee. 

INGE CUC Journal respects the moral rights of its authors, whom must cede the editorial committee the patrimonial rights of the published material. In turn, the authors inform that the current work is unpublished and has not been previously published. 

All articles are licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Licencia Creative Commons