I’m an academic and a software developer. Studies of software developers suggest that practising developers typically refer to peers, books, blogs, and other social media rather than academic research when looking to solve problems (Beecham et al, 2014; Petersen et al, 2014). My own experience of software development tells me that the former sorts of resources can be very effective for day-to-day problems such as diagnosing a particular error message or needing to know the syntax of an unfamiliar programming language. So what is the role of scientific research in software development?
Large software development houses like Microsoft and Google operate their own research labs, looking for ways to give their existing products an edge over competitors or to develop new products. But even smaller companies have reason to improve their products and their processes beyond day-to-day coding and bug-finding. In a study of small-to-medium enterprises in Denmark, Sarai Løkkegard (2019) observes “the situation of SMEs is that they, through a constant development of ideas, products and solutions, continuously have to justify their existence”. The participants in her study sought new knowledge for updating their skills, inspiration for new products, and product development, and many (though not all) recognised that scientific knowledge could assist with these. Like the aforementioned software developers, however, participants in the study rarely made use of academic research.
Løkkegard reports that lack of time was the by far the most common barrier to seeking out new knowledge, and for this reason SMEs did not search research publications for inspiration as much as they might like. SMEs also found seeking new knowledge to be expensive, turning it into something useful to be difficult, and the amount of research available to be overwhelming. A Little Research attempts to mitigate all of these problems by identifying the research most relevant to a particular problem and translating it into a form in which it can be understood by practitioners, with lower overheads than setting up a full-scale research lab or university project.
What do software developers want from academic research?
The ACM’s Special Interest Group on Software Engineering (SIGSOFT) established the Impact Project around 2001 with the intention “to determine the impact of software engineering research upon software engineering practice”. Leon Osterweil and colleagues (2008) summarised the findings as:
- Software engineering research has significantly affected software engineering practice.
- Lasting impact seems to come most readily from ongoing interactions between research and practice.
- Research impact might not be fully felt for at least 10 years.
- Continued support for sustaining a vigorous research community is required.
- A research community must provide nurturing in a variety of ways.
- The interplay between research and practice can be difficult to determine precisely and communicate clearly.
- It seems more beneficial to identify the many and varied ways in which research and practice have influenced each other than to seek precise criteria for determining what should be considered “research impact”.
More recently, Vahid Garousi and colleagues (2018) conducted a wide-ranging review of the relevance of software engineering research to practice. They say that calls for research to be more “relevant” have been made since (at least) the 1980s but that much academic research is still not perceived to be relevant to practitioners (though they also make the point that basic research intended to understand how software works is a source of innovation and re-iterate the Impact Project’s point that it can take 10-20 years for research findings to be integrated into practice). Based on a review of 46 papers discussing the topic, Garousi and colleagues suggest three main causes for a lack of relevance of academic research:
- Simplistic or incorrect assumptions about software engineering practice.
- Incorrect identification of problems to be resolved by research.
- Mismatches between researchers’ mindsets as to how research ought to be done and how software engineering is carried out in practice.
They suggest that these problems could be combatted by using alternative research approaches, collaboration with industry, and the selection of “practical” problems for research. For my part, I hope A Little Research provides a link between the research and practice communities, finds that research that is relevant to a particular organisation, and gets it into practice as quickly as possible.
Finally, Vladimir Ivanov and colleagues (2017) asked 67 software developers from a variety of firms in Innopolis, Russia, “what they cared about when developing software”. Their respondents said that their most pressing issue was accurate estimation of the the amount of effort required to complete software development, and that they cared more about timely delivery of software than about its quality. Ivanov and colleagues note that methods for estimating effort do actually exist in the academic literature, but that the survey respondents appeared to be unaware of them. When asked what they most wanted to know about, the developers in fact nominated DevOps and testing techniques—apparently the developers thought to learn more about techniques of which they were already aware (DevOps and testing) but they did not even know to search for techniques that would solve the problem they most commonly experienced. A Little Research exists to find just this sort of information, that developers won’t find by searching for terms that they already know about.
Communicating and making use of research
Deepika Badampudi and colleagues (2019) propose a knowledge transfer process for software engineering based on Bayesian synthesis approaches used in healthcare. They describe the transfer process as a cycle but we can imagine the process starting at the point at which practitioners come up against a problem they would like to solve:
- Identify, review and select knowledge that might address the needs identified by the practitioners.
- Elicit prior probability, that is, establish the current practices practices of those likely to be involved in addressing the need.
- Assess gaps in the skills and knowledge currently possessed by the practitioners relative to the skills and knowledge desired of them.
- Align external knowledge to local context by reviewing the value and applicability of the external knowledge to the practitioners’ context, and adapting terminology where necessary.
- Assess barriers and facilitators to knowledge use.
- Select, tailor and implement knowledge translation intervention in light of the foregoing assessments.
- Monitor use.
- Evaluate outcomes.
- Sustain use.
A Little Research provides literature search and review services intended to satisfy the identification and review step, in the manner of the “rapid reviews” and “evidence briefings” described by Bruno Cartaxo and colleagues (2018) in the context of software engineering. A Little Research can also help with some of the later steps in the transfer process, such as recommending or providing training, and gathering and evaluating evidence. Please refer to our home page for a summary of our services.
Deepika Badampudi, Claes Wohlin, and Tony Gorschek 2019. Contextualizing Research Evidence through Knowledge Translation in Software Engineering. Evaluation and Assessment in Software Engineering, Copenhagen, Denmark, April 15–17, 2019.
Sarah Beecham, Pádraig O’Leary, Sara Baker, Ita Richardson and John Noll 2014. Making Software Engineering Research Relevant. IEEE Computer 47(4), April 2014, pages 80-83.
Bruno Cartaxo, Gustavo Pinto and Sergio Soares 2018. Towards a Model to Transfer Knowledge From Software Engineering Research to Practice. Information and Software Technology 97, pages 80-82.
Vahid Garousi, Markus Borg and Tony Gorschek 2018. Cut to the Chase: Revisiting the Relevance of Software Engineering Research. Unpublished manuscript available from arXiv, https://arxiv.org/abs/1812.01395.
Vladimir Ivanov, Alan Rogers, Giancarlo Succi, Jooyong Yi and Vaslii Zorin 2017. What Do Software Engineers Care About? Gaps between Research and Practice. Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, Paderborn, Germany.
Sarai Løkkegaard 2018. Disseminating Scientific Knowledge to Small and Medium-sized Enterprises. Ph.D.-Serien, Det Humanistiske Fakultet, Aalborg Universitet.
Leon J. Osterweil, Carlo Ghezzi, Jeff Kramer and Alexander L. Wolf 2008. Determining the Impact of Software Engineering Research on Practice. IEEE Computer 41(3), March 2008, pages 39-49.
Kai Petersen and Emilie Engström. Finding Relevant Research Solutions for Practical Problems — the SERP Taxonomy Architecture. International Workshop on Long-Term Industrial Collaboration in Software Engineering, Västeras, Sweden, 2014.