https://dl.acm.org/doi/10.1007/s10664-017-9579-0 ACM Digital Library home ACM home * Advanced Search * Browse * About * + Sign in + Register * * Advanced Search * Journals * Magazines * Proceedings * Books * SIGs * Conferences * People * * More * Search ACM Digital Library[ ] SearchSearch Advanced Search Empirical Software Engineering * Periodical Home * Latest Issue * * Archive * Authors * Affiliations * Award Winners * More HomeBrowse by TitlePeriodicalsEmpirical Software EngineeringVol. 23, No. 5Does syntax highlighting help programming novices? article Free Access Share on Does syntax highlighting help programming novices? * Authors: * #Christoph Hannebauer paluno --- The Ruhr Institute for Software Technology, University of Duisburg-Essen, Essen, Germany paluno --- The Ruhr Institute for Software Technology, University of Duisburg-Essen, Essen, Germany View Profile , * #Marc Hesenius paluno --- The Ruhr Institute for Software Technology, University of Duisburg-Essen, Essen, Germany paluno --- The Ruhr Institute for Software Technology, University of Duisburg-Essen, Essen, Germany View Profile , * #Volker Gruhn paluno --- The Ruhr Institute for Software Technology, University of Duisburg-Essen, Essen, Germany paluno --- The Ruhr Institute for Software Technology, University of Duisburg-Essen, Essen, Germany View Profile Authors Info & Claims Empirical Software EngineeringVolume 23Issue 5October 2018 pp 2795-2828https://doi.org/10.1007/s10664-017-9579-0 Online:01 October 2018Publication History * 2citation * 0 * Downloads Metrics Total Citations2 Total Downloads0 Last 12 Months0 Last 6 weeks0 * * Get Citation Alerts New Citation Alert added! This alert has been successfully added and will be sent to: You will be notified whenever a record that you have chosen has been cited. To manage your alert preferences, click on the button below. Manage my Alerts New Citation Alert! Please log in to your account * Save to Binder Save to Binder [loader] Create a New Binder Name [ ] + Cancel + Create * Export Citation * Publisher Site * Empirical Software Engineering Volume 23, Issue 5 PreviousArticleNextArticle ACM Digital Library Abstract Program comprehension is an important skill for programmers --- extending and debugging existing source code is part of the daily routine. Syntax highlighting is one of the most common tools used to support developers in understanding algorithms. However, most research in this area originates from a time when programmers used a completely different tool chain. We examined the influence of syntax highlighting on novices' ability to comprehend source code. Additional analyses cover the influence of task type and programming experience on the code comprehension ability itself and its relation to syntax highlighting. We conducted a controlled experiment with 390 undergraduate students in an introductory Java programming course. We measured the correctness with which they solved small coding tasks. Each test subject received some tasks with syntax highlighting and some without. The data provided no evidence that syntax highlighting improves novices' ability to comprehend source code. There are very few similar experiments and it is unclear as of yet which factors impact the effectiveness of syntax highlighting. One major limitation may be the types of tasks chosen for this experiment. The results suggest that syntax highlighting squanders a feedback channel from the IDE to the programmer that can be used more effectively. References 1. Allen E, Cartwright R, Stoler B (2002) Drjava: a lightweight pedagogic environment for java. ACM SIGCSE Bullet. 34(1):137. Google ScholarGoogle ScholarCross RefCross Ref 2. Baecker RM, Marcus A (1989) Human factors and typography for more readable programs. ACM, New York. Google ScholarGoogle Scholar 3. Beelders TR, du Plessis JPL (2015) Syntax highlighting as an influencing factor when reading and comprehending source code. J Eye Mov Res 9(1).Google ScholarGoogle Scholar 4. Bergin S, Reilly R (2005) Programming: Factors that influence success. SIGCSE Bull 37(1):411-415. Google ScholarGoogle Scholar Digital LibraryDigital Library 5. Bird C, Nagappan N, Murphy B, Gall H, Devanbu P (2011) Don't touch my code!: Examining the effects of ownership on software quality. In: Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering, ACM, New York, ESEC/FSE '11, pp 4-14. Google Scholar Google Scholar 6. Chaffin WW, Rhiel SG (1993) The effect of skewness and kurtosis on the one-sample t test and the impact of knowledge of the population standard deviation. J Stat Comput Simul 46(1-2):79-90. Google ScholarGoogle Scholar 7. Crosby M, Stelovsky J (1990) How do we read algorithms? a case study. Computer 23(1):25-35. Google ScholarGoogle ScholarDigital LibraryDigital Library 8. Dieste O, Aranda AM, Uyaguari F, Turhan B, Tosun A, Fucci D, Oivo M, Juristo N (2017) Empirical evaluation of the effects of experience on code quality and programmer productivity: an exploratory study. Empir Softw Eng 22:1-86. ISSN:1573-7616. Google ScholarGoogle Scholar 9. Dimitri GM (2015) The impact of syntax highlighting in sonic pi. In: Psychology of Programming Interest Group Annual Conference 2015. Proceedings of, pp 59-70.Google ScholarGoogle Scholar 10. Dragicevic P (2016) Fair Statistical Communication in HCI. Springer International Publishing, Cham, pp 291-330.Google ScholarGoogle Scholar 11. Erguler K (2013) Barnard: Barnard's Unconditional Test. R package version 1.3. http://cran.r-project.org/package=Barnard, [accessed 2014-07-22].Google ScholarGoogle Scholar 12. Feigenspan J, Kastner C, Apel S, Liebig J, Schulze M, Dachselt R, Papendieck M, Leich T, Saake G (2013) Do background colors improve program comprehension in the #ifdef hell? Empir Softw Eng 18(4):699- 745. Google ScholarGoogle ScholarDigital Library Digital Library 13. Gellenbeck EM, Cook CR (1991) Does signaling help professional programmers read and understand computer programs? In: Fourth Workshop on Empirical Studies of Programmers, Ablex Publishing Corporation, Norwood, pp 82-98.Google ScholarGoogle Scholar 14. Goldstein EB (1995) Sensation and Perception. Wadsworth Publishing, Belmont.Google ScholarGoogle Scholar 15. Grant EE, Sackman H (1967) An exploratory investigation of programmer performance under on-line and off-line conditions. IEEE Trans Hum Fact Electron HFE-8(1):33-48.Google ScholarGoogle Scholar 16. Gray KE, Flatt M (2003) ProfessorJ: A gradual introduction to java through language levels. In: Companion of the 18th Annual ACM SIGPLAN Conference on Object-oriented Programming, Systems, Languages, and Applications, ACM Press, New York, OOPSLA '03, pp 170-177. Google ScholarGoogle Scholar 17. Gruhn V, Hannebauer C (2012) Using wikis as software development environments. In: Proceedings of the the 11th SoMeT 12, IOS Press, Amsterdam, Frontiers in Artificial Intelligence and Applications, vol 246, pp 3-20.Google ScholarGoogle Scholar 18. Hakala T, Nykyri P, Sajaniemi J (2006) An experiment on the effects of program code highlighting on visual search for local patterns. In: 18th Workshop of the Psychology of Programming Interest Group, Proceedings of, pp 38-52.Google ScholarGoogle Scholar 19. Hannebauer C, Hesenius M, Gruhn V (2017) Lab package for the syntax highlighting experiment. https://www.uni-due.de/~hw0433/sh /.Google ScholarGoogle Scholar 20. Holden E, Weeden E (2003) The impact of prior experience in an information technology programming course sequence. In: Proceedings of the 4th Conference on Information Technology Curriculum, ACM, New York, CITC4 '03, pp 41-46. Google Scholar Google Scholar 21. Jedlitschka A, Ciolkowski M, Pfahl D (2008) Reporting experiments in software engineering. Springer, Berlin, pp 201-228.Google ScholarGoogle Scholar 22. Kitchenham B, Pfleeger S, Pickard L, Jones P, Hoaglin D, El Emam K, Rosenberg J (2002) Preliminary guidelines for empirical research in software engineering. IEEE Trans Softw Eng 28 (8):721-734. Google ScholarGoogle ScholarDigital LibraryDigital Library 23. Kleinschmager S, Hanenberg S (2011) How to rate programming skills in programming experiments?: A preliminary, exploratory, study based on university marks, pretests, and self-estimation. In: Proceedings of the 3rd ACM SIGPLAN Workshop on Evaluation and Usability of Programming Languages and Tools, ACM, New York, PLATEAU '11, pp 15-24. Google ScholarGoogle Scholar 24. Knuth DE (1984) Literate programming. Comput J 27(2):97-111. Google ScholarGoogle ScholarDigital LibraryDigital Library 25. van Laar D (1989) Evaluating a colour coding programming support tool. In: Sutcliffe A, Macaulay L (eds) Proceedings of the Fifth Conference of the British Computer Society Human-Computer Interaction Specialist Group, Cambridge University Press. British Computer Society Workshop Series, Cambridge, pp 217-230. Google ScholarGoogle Scholar 26. Lientz BP, Swanson EB, Tompkins GE (1978) Characteristics of application software maintenance. Commun ACM 21(6):466-471. Google ScholarGoogle ScholarDigital LibraryDigital Library 27. LPLUS GmbH (2014) LPLUS-System on-campus. http://lplus.de/ lplus.de/index_9_19_2_.html, [accessed 2014-07-16].Google Scholar Google Scholar 28. Ludbrook J (2008) Analysis of 2 x 2 tables of frequencies: matching test to experimental design. Int J Epidemiol 37 (6):1430-1435.Google ScholarGoogle Scholar 29. Mehta R, Zhu RJ (2009) Blue or red? exploring the effect of color on cognitive task performances. Science 323(5918):1226-1229. Google ScholarGoogle Scholar 30. Microsoft Developer Network (2015) C# coding conventions (c# programming guide). http://msdn.microsoft.com/en-us/library/ ff926074.aspx, [accessed 2017-03-08].Google ScholarGoogle Scholar 31. Oman PW, Cook CR (1990) Typographic style is more than cosmetic, vol 33. Google ScholarGoogle Scholar 32. Oracle Technology Network (1999) Code conventions for the java programming language. http://www.oracle.com/technetwork/java/ javase/documentation/codeconvtoc-136057.html, [accessed 2017-03-08].Google ScholarGoogle Scholar 33. Prechelt L (1999) The 28:1 Grant/Sackman legend is misleading, or: How large is interpersonal variation really? Interner Bericht, Fakultat fur Informatik. Universitat Karlsruhe, Karlsruhe.Google ScholarGoogle Scholar 34. Rambally GK (1986) The influence of color on program readability and comprehensibility. In: Proceedings of the Seventeenth SIGCSE Technical Symposium on Computer Science Education, ACM, New York, SIGCSE '86, pp 173-181. Google ScholarGoogle Scholar 35. Reijers H, Freytag T, Mendling J, Eckleder A (2011) Syntax highlighting in business process models. Decis Support Syst 51 (3):339-349. Google ScholarGoogle ScholarDigital LibraryDigital Library 36. Sackman H, Erikson WJ, Grant EE (1968) Exploratory experimental studies comparing online and offline programming performance. Commun ACM 11(1):3-11. Google ScholarGoogle ScholarDigital LibraryDigital Library 37. Sarkar A (2015) The impact of syntax colouring on program comprehension. In: Psychology of Programming Interest Group Annual Conference 2015, Proceedings of, pp 49-58.Google Scholar Google Scholar 38. Scintilla Project (2014) Scintilla - a free source code editing component for win32, gtk+, and os x. http://www.scintilla.org, [accessed 2014-07-22].Google ScholarGoogle Scholar 39. Seo H, Sadowski C, Elbaum S, Aftandilian E, Bowdidge R (2014) Programmers' build errors: A case study (at google). In: Proceedings of the 36th International Conference on Software Engineering, ACM, New York, ICSE 2014, pp 724-734. Google Scholar Google Scholar 40. Siegmund J, Kastner C, Liebig J, Apel S, Hanenberg S (2014) Measuring and modeling programming experience. Empir Softw Eng 19 (5):1299-1334. Google ScholarGoogle ScholarDigital LibraryDigital Library 41. Tapp R, Kazman R (1994) Determining the usefulness of colour and fonts in a programming task. In: Proceedings of the Third IEEE Workshop on Program Comprehension. IEEE Computer Society Press, Los Alamitos, pp 154-161.Google ScholarGoogle Scholar 42. Tiarks R (2011) What programmers really do: an observational study. Softwaretechnik-Trends 31(2):36-37.Google ScholarGoogle Scholar 43. Wallis WA (1942) Compounding probabilities from independent significance tests. Econometrica, J Econ Soc 10(3/4):229-248. Google ScholarGoogle ScholarCross RefCross Ref Comments Please enable JavaScript to view thecomments powered by Disqus. Login options Check if you have access through your login credentials or your institution to get full access on this article. Sign in Full Access Get this Article * Information * Contributors * Published in Empirical Software Engineering cover image Empirical Software Engineering Volume 23, Issue 5 October 2018 551 pages ISSN:1382-3256 Issue's Table of Contents Copyright (c) Copyright (c) 2018 Springer Science+Business Media, LLC, part of Springer Nature Sponsors In-Cooperation Publisher Kluwer Academic Publishers United States Publication History + Online: 1 October 2018 Author Tags + IDE interface + Source code typography + Code colouring + Program comprehension + Syntax highlighting Qualifiers + article Conference Funding Sources * [loader] Other Metrics View Article Metrics * Bibliometrics * Citations2 * Article Metrics + 2 Total Citations View Citations + 0 Total Downloads + Downloads (Last 12 months)0 + Downloads (Last 6 weeks)0 Other Metrics View Author Metrics * Cited By View all Digital Edition View this article in digital edition. View Digital Edition * Figures * Other * * Share this Publication link https://dl.acm.org/doi/10.1007/s10664-017-9579-0 Copy Link Share on Social Media Share on * * * * 0References * * * Close Figure Viewer Browse AllReturnChange zoom level[ ] Caption View Issue's Table of Contents Export Citations Select Citation format[BibTeX ] + Download citation + Copy citation * Preview is not available. By clicking download,a new tab will open to start the export process. The process may takea few minutes but once it finishes a file will be downloaded on your browser soplease do not close the new tab. Download Categories * Journals * Magazines * Books * Proceedings * SIGs * Conferences * Collections * People About * About ACM Digital Library * ACM Digital Library Board * Subscription Information * Author Guidelines * Using ACM Digital Library * All Holdings within the ACM Digital Library * ACM Computing Classification System * Digital Library Accessibility Join * Join ACM * Join SIGs * Subscribe to Publications * Institutions and Libraries Connect * Contact * Facebook * Twitter * Linkedin The ACM Digital Library is published by the Association for Computing Machinery. Copyright (c) 2022 ACM, Inc. * Terms of Usage * Privacy Policy * Code of Ethics ACM Digital Library home ACM home About Cookies On This Site We use cookies to ensure that we give you the best experience on our website. Learn more Got it!