It Will Never Work in Theory

Greg Wilson,Jorge Aranda,Michael Hoye,Brittany Johnson

DOI: https://doi.org/10.1109/ms.2024.3362649

IF: 3

2024-01-01

IEEE Software

Abstract:We have been trying to get software engineering researchers and practitioners to talk to one another for over a decade. This paper describes what we have done, assesses our impact, and recommends an approach that we hope will have greater success.

computer science, software engineering

What problem does this paper attempt to address?

It Will Never Work in Theory

Greg Wilson,Jorge Aranda,Michael Hoye,Brittany Johnson

2024-02-16

Abstract:We have been trying to get software engineering researchers and practitioners to talk to one another for over a decade. This paper describes what we have done, assesses our impact, and recommends an approach that we hope will have greater success.

Software Engineering
Philosophical Smoke Signals: Theory and Practice in Information Systems Design

David King,Chris Kimble

DOI: https://doi.org/10.48550/arXiv.0804.2852

2008-04-17

Software Engineering

Abstract:Although the gulf between the theory and practice in Information Systems is much lamented, few researchers have offered a way forward except through a number of (failed) attempts to develop a single systematic theory for Information Systems. In this paper, we encourage researchers to re-examine the practical consequences of their theoretical arguments. By examining these arguments we may be able to form a number of more rigorous theories of Information Systems, allowing us to draw theory and practice together without undertaking yet another attempt at the holy grail of a single unified systematic theory of Information Systems.
Theory-Software Translation: Research Challenges and Future Directions

Caroline Jay,Robert Haines,Daniel S. Katz,Jeffrey Carver,James C. Phillips,Anshu Dubey,Sandra Gesing,Matthew Turk,Hui Wan,Hubertus van Dam,James Howison,Vitali Morozov,Steven R. Brandt

DOI: https://doi.org/10.48550/arXiv.1910.09902

2019-10-22

Software Engineering

Abstract:The Theory-Software Translation Workshop, held in New Orleans in February 2019, explored in depth the process of both instantiating theory in software - for example, implementing a mathematical model in code as part of a simulation - and using the outputs of software - such as the behavior of a simulation - to advance knowledge. As computation within research is now ubiquitous, the workshop provided a timely opportunity to reflect on the particular challenges of research software engineering - the process of developing and maintaining software for scientific discovery. In addition to the general challenges common to all software development projects, research software additionally must represent, manipulate, and provide data for complex theoretical constructs. Ensuring this process is robust is essential to maintaining the integrity of the science resulting from it, and the workshop highlighted a number of areas where the current approach to research software engineering would benefit from an evidence base that could be used to inform best practice. The workshop brought together expert research software engineers and academics to discuss the challenges of Theory-Software Translation over a two-day period. This report provides an overview of the workshop activities, and a synthesises of the discussion that was recorded. The body of the report presents a thematic analysis of the challenges of Theory-Software Translation as identified by workshop participants, summarises these into a set of research areas, and provides recommendations for the future direction of this work.
An Experience Report On Applying Software Testing Academic Results In Industry: We Need Usable Automated Test Generation

Andrea Arcuri

DOI: https://doi.org/10.48550/arXiv.1901.03865

2019-01-12

Software Engineering

Abstract:What is the impact of software engineering research on current practices in industry? In this paper, I report on my direct experience as a PhD/post-doc working in software engineering research projects, and then spending the following five years as an engineer in two different companies (the first one being the same I worked in collaboration with during my post-doc). Given a background in software engineering research, what cutting-edge techniques and tools from academia did I use in my daily work when developing and testing the systems of these companies? Regarding validation and verification (my main area of research), the answer is rather short: as far as I can tell, only FindBugs. In this paper, I report on why this was the case, and discuss all the challenging, complex open problems we face in industry and which somehow are "neglected" in the academic circles. In particular, I will first discuss what actual tools I could use in my daily work, such as JaCoCo and Selenium. Then, I will discuss the main open problems I faced, particularly related to environment simulators, unit and web testing. After that, popular topics in academia are presented, such as UML, regression and mutation testing. Their lack of impact on the type of projects I worked on in industry is then discussed. Finally, from this industrial experience, I provide my opinions about how this situation can be improved, in particular related to how academics are evaluated, and advocate for a greater involvement into open-source projects.
Social Science Theories in Software Engineering Research

Tobias Lorey,Paul Ralph,Michael Felderer

DOI: https://doi.org/10.48550/arXiv.2202.07519

2022-02-15

Abstract:As software engineering research becomes more concerned with the psychological, sociological and managerial aspects of software development, relevant theories from reference disciplines are increasingly important for understanding the field's core phenomena of interest. However, the degree to which software engineering research draws on relevant social sciences remains unclear. This study therefore investigates the use of social science theories in five influential software engineering journals over 13 years. It analyzes not only the extent of theory use but also what, how and where these theories are used. While 87 different theories are used, less than two percent of papers use a social science theory, most theories are used in only one paper, most social sciences are ignored, and the theories are rarely tested for applicability to software engineering contexts. Ignoring relevant social science theories may (1) undermine the community's ability to generate, elaborate and maintain a cumulative body of knowledge; and (2) lead to oversimplified models of software engineering phenomena. More attention to theory is needed for software engineering to mature as a scientific discipline.

Software Engineering
Improving software practice through education: Challenges and future trends

Timothy C. Lethbridge,Jorge Díaz-Herrera,Richard J. LeBlanc,J. Barrie Thompson,Jorge Diaz-Herrera,Richard J. Jr. LeBlanc

DOI: https://doi.org/10.1109/fose.2007.13

2007-05-01

Abstract:We argue that the software engineering (SE) community could have a significant impact on the future of the discipline by focusing its efforts on improving the education of software engineers. There are some bright spots such as the various projects to codify knowledge, and the development of undergraduate SE programs. However, there remain several key challenges, each of which is addressed in this paper: The challenges are 1) making programs attractive to students, 2) focusing education appropriately, 3) communicating industrial reality more effectively, 4) defining curricula that are forward-looking, 5) providing education for existing practitioners, 6) making SE education more evidence-based, 7) ensuring that SE educators have the necessary background, and 8) raising the prestige and quality of SE educational research. For each challenge, we provide action items and open research questions.
Industry-academia research collaboration and knowledge co-creation: Patterns and anti-patterns

Dusica Marijan,Sagar Sen

DOI: https://doi.org/10.1145/3494519

2022-04-30

Abstract:Increasing the impact of software engineering research in the software industry and the society at large has long been a concern of high priority for the software engineering community. The problem of two cultures, research conducted in a vacuum (disconnected from the real world), or misaligned time horizons are just some of the many complex challenges standing in the way of successful industry-academia collaborations. This paper reports on the experience of research collaboration and knowledge co-creation between industry and academia in software engineering as a way to bridge the research-practice collaboration gap. Our experience spans 14 years of collaboration between researchers in software engineering and the European and Norwegian software and IT industry. Using the participant observation and interview methods we have collected and afterwards analyzed an extensive record of qualitative data. Drawing upon the findings made and the experience gained, we provide a set of 14 patterns and 14 anti-patterns for industry-academia collaborations, aimed to support other researchers and practitioners in establishing and running research collaboration projects in software engineering.

Software Engineering,Machine Learning
Roundtable: The Future of Software Engineering for Internet Computing

Jian Lu,David S. Rosenblum,Tevfik Bultan,Valérie Issarny,Schahram Dustdar,Margaret-Anne D. Storey,Dongmei Zhang

DOI: https://doi.org/10.1109/MS.2015.15

IF: 3

2015-01-01

IEEE Software

Abstract:Seven research leaders in software engineering for Internet computing present their viewpoints on important issues that will shape this field's future. They discuss opportunities and challenges for the shifting software paradigm; stepping outside the comfort zone to revisit issues such as software correctness; improving Internet software dependability and programmability; addressing software engineering issues for the Internet of Things; exploring relationships among the Internet of Things, people, and software services; supporting a participatory culture of software development; and rethinking logging in online services.
Industry-Academia Research Collaboration in Software Engineering: The Certus Model

Dusica Marijan,Arnaud Gotlieb

DOI: https://doi.org/10.48550/arXiv.2204.11039

2022-04-23

Software Engineering

Abstract:Context: Research collaborations between software engineering industry and academia can provide significant benefits to both sides, including improved innovation capacity for industry, and real-world environment for motivating and validating research ideas. However, building scalable and effective research collaborations in software engineering is known to be challenging. While such challenges can be varied and many, in this paper we focus on the challenges of achieving participative knowledge creation supported by active dialog between industry and academia and continuous commitment to joint problem solving. Objective: This paper aims to understand what are the elements of a successful industry-academia collaboration that enable the culture of participative knowledge creation. Method: We conducted participant observation collecting qualitative data spanning 8 years of collaborative research between a software engineering research group on software V&V and the Norwegian IT sector. The collected data was analyzed and synthesized into a practical collaboration model, named the Certus Model. Results: The model is structured in seven phases, describing activities from setting up research projects to the exploitation of research results. As such, the Certus model advances other collaborations models from literature by delineating different phases covering the complete life cycle of participative research knowledge creation. Conclusion: The Certus model describes the elements of a research collaboration process between researchers and practitioners in software engineering, grounded on the principles of research knowledge co-creation and continuous commitment to joint problem solving. The model can be applied and tested in other contexts where it may be adapted to the local context through experimentation.
A Community's Perspective on the Status and Future of Peer Review in Software Engineering

Lutz Prechelt,Daniel Graziotin,Daniel Méndez Fernández

DOI: https://doi.org/10.48550/arXiv.1706.07196

2017-10-30

Abstract:Context: Pre-publication peer review of scientific articles is considered a key element of the research process in software engineering, yet it is often perceived as not to work fully well. Objective: We aim at understanding the perceptions of and attitudes towards peer review of authors and reviewers at one of software engineering's most prestigious venues, the International Conference on Software Engineering (ICSE). Method: We invited 932 ICSE 2014/15/16 authors and reviewers to participate in a survey with 10 closed and 9 open questions. Results: We present a multitude of results, such as: Respondents perceive only one third of all reviews to be good, yet one third as useless or misleading; they propose double-blind or zero-blind reviewing regimes for improvement; they would like to see showable proofs of (good) reviewing work be introduced; attitude change trends are weak. Conclusion: The perception of the current state of software engineering peer review is fairly negative. Also, we found hardly any trend that suggests reviewing will improve by itself over time; the community will have to make explicit efforts. Fortunately, our (mostly senior) respondents appear more open for trying different peer reviewing regimes than we had expected.

Software Engineering,Computers and Society,Digital Libraries
Towards a Philosophy of Software Development: 40 Years after the Birth of Software Engineering

Mandy Northover,Derrick G. Kourie,Andrew Boake,Stefan Gruner,Alan Northover

DOI: https://doi.org/10.1007/s10838-008-9068-7

2008-08-06

Journal for General Philosophy of Science

Abstract:Over the past four decades, software engineering has emerged as a discipline in its own right, though it has roots both in computer science and in classical engineering. Its philosophical foundations and premises are not yet well understood. In recent times, members of the software engineering community have started to search for such foundations. In particular, the philosophies of Kuhn and Popper have been used by philosophically-minded software engineers in search of a deeper understanding of their discipline. It seems, however, that professional philosophers of science are not yet aware of this new discourse within the field of software engineering. Therefore, this article aims to reflect critically upon recent software engineers’ attempts towards a philosophy of software engineering and to introduce our own philosophical thoughts in this context. Finally, we invite the professional philosophers of science to participate in this interesting new discourse.
Four presumed gaps in the software engineering research community's knowledge

Lutz Prechelt

DOI: https://doi.org/10.48550/arXiv.1911.09971

2019-11-22

Software Engineering

Abstract:Background: The state of the art in software engineering consists of a myriad of contributions and the gaps between them; it is difficult to characterize. Questions: In order to help understanding the state of the art, can we identify gaps in our knowledge that are at a very general, widely relevant level? Which research directions do these gaps suggest? Method: 54 expert interviews with senior members of the ICSE community, evaluated qualitatively using elements of Grounded Theory Methodology. Results: Our understanding of complexity, of good-enoughness, and of developers' strengths is underdeveloped. Some other relevant factors' relevance is apparently not clear. Software engineering is not yet an evidence-based discipline. Conclusion: More software engineering research should concern itself with emergence phenomena, with how engineering tradeoffs are made, with the assumptions underlying research works, and with creating certain taxonomies. Such work would also allow software engineering to become more evidence-based.
Software Citation in Theory and Practice

Daniel S. Katz,Neil P. Chue Hong

DOI: https://doi.org/10.1007/978-3-319-96418-8_34

2018-07-21

Abstract:In most fields, computational models and data analysis have become a significant part of how research is performed, in addition to the more traditional theory and experiment. Mathematics is no exception to this trend. While the system of publication and credit for theory and experiment (journals and books, often monographs) has developed and has become an expected part of the culture, how research is shared and how candidates for hiring, promotion are evaluated, software (and data) do not have the same history. A group working as part of the FORCE11 community developed a set of principles for software citation that fit software into the journal citation system, allow software to be published and then cited, and there are now over 50,000 DOIs that have been issued for software. However, some challenges remain, including: promoting the idea of software citation to developers and users; collaborating with publishers to ensure that systems collect and retain required metadata; ensuring that the rest of the scholarly infrastructure, particularly indexing sites, include software; working with communities so that software efforts "count" and understanding how best to cite software that has not been published.

Computers and Society
Software engineering and the SP Theory of Intelligence

J Gerard Wolff

DOI: https://doi.org/10.48550/arXiv.1708.06665

2017-08-18

Software Engineering

Abstract:This paper describes a novel approach to software engineering derived from the "SP Theory of Intelligence" and its realisation in the "SP Computer Model". Despite superficial appearances, it is shown that many of the key ideas in software engineering have counterparts in the structure and workings of the SP system. Potential benefits of this new approach to software engineering include: the automation or semi-automation of software development, with support for programming of the SP system where necessary; allowing programmers to concentrate on 'world-oriented' parallelism, without worries about parallelism to speed up processing; support for the long-term goal of programming the SP system via written or spoken natural language; reducing or eliminating the distinction between 'design' and 'implementation'; reducing or eliminating operations like compiling or interpretation; reducing or eliminating the need for verification of software; reducing the need for validation of software; no formal distinction between program and database; the potential for substantial reductions in the number of types of data file and the number of computer languages; benefits for version control; and reducing technical debt.
Using grounded theory to study the experience of software development

Steve Adolph,Wendy Hall,Philippe Kruchten

DOI: https://doi.org/10.1007/s10664-010-9152-6

IF: 3.762

2011-01-27

Empirical Software Engineering

Abstract:Grounded Theory is a research method that generates theory from data and is useful for understanding how people resolve problems that are of concern to them. Although the method looks deceptively simple in concept, implementing Grounded Theory research can often be confusing in practice. Furthermore, despite many papers in the social science disciplines and nursing describing the use of Grounded Theory, there are very few examples and relevant guides for the software engineering researcher. This paper describes our experience using classical (i.e., Glaserian) Grounded Theory in a software engineering context and attempts to interpret the canons of classical Grounded Theory in a manner that is relevant to software engineers. We provide model to help the software engineering researchers interpret the often fuzzy definitions found in Grounded Theory texts and share our experience and lessons learned during our research. We summarize these lessons learned in a set of fifteen guidelines.

computer science, software engineering
The Four Pillars of Research Software Engineering

J. Cohen,D. S. Katz,M. Barker,N. Chue Hong,R. Haines,C. Jay

DOI: https://doi.org/10.1109/MS.2020.2973362

2023-01-26

Abstract:Building software that can support the huge growth in data and computation required by modern research needs individuals with increasingly specialist skill sets that take time to develop and maintain. The Research Software Engineering movement, which started in the UK and has been built up over recent years, aims to recognise and support these individuals. Why does research software matter to professional software development practitioners outside the research community? Research software can have great impact on the wider world and recent progress means the area can now be considered as a more realistic option for a professional software development career. In this article we present a structure, along with supporting evidence of real-world activities, that defines four elements that we believe are key to providing comprehensive and sustainable support for Research Software Engineering. We also highlight ways that the wider developer community can learn from, and engage with, these activities.

Software Engineering
The Essence Theory of Software Engineering - Large-Scale Classroom Experiences from 450+ Software Engineering BSc Students

Kai-Kristian Kemell,Anh Nguyen-Duc,Xiaofeng Wang,Juhanki Risku,Pekka Abrahamsson

DOI: https://doi.org/10.48550/arXiv.1809.08827

2018-09-24

Software Engineering

Abstract:Software Engineering as an industry is highly diverse in terms of development methods and practices. Practitioners employ a myriad of methods and tend to further tailor them by e.g. omitting some practices or rules. This diversity in development methods poses a challenge for software engineering education, creating a gap between education and industry. General theories such as the Essence Theory of Software Engineering can help bridge this gap by presenting software engineering students with higher-level frameworks upon which to build an understanding of software engineering methods and practical project work. In this paper, we study Essence in an educational setting to evaluate its usefulness for software engineering students while also investigating barriers to its adoption in this context. To this end, we observe 102 student teams utilize Essence in practical software engineering projects during a semester long, project-based course.
Technical Dimensions of Programming Systems

Joel Jakubovic,Jonathan Edwards,Tomas Petricek

DOI: https://doi.org/10.48550/arXiv.2302.10003

2023-02-20

Programming Languages

Abstract:Programming requires much more than just writing code in a programming language. It is usually done in the context of a stateful environment, by interacting with a system through a graphical user interface. Yet, this wide space of possibilities lacks a common structure for navigation. Work on programming systems fails to form a coherent body of research, making it hard to improve on past work and advance the state of the art. In computer science, much has been said and done to allow comparison of **programming languages**, yet no similar theory exists for *programming systems*; we believe that programming systems deserve a theory too. We present a framework of *technical dimensions* which capture the underlying characteristics of programming systems and provide a means for conceptualizing and comparing them. We identify technical dimensions by examining past influential programming systems and reviewing their design principles, technical capabilities, and styles of user interaction. Technical dimensions capture characteristics that may be studied, compared and advanced independently. This makes it possible to talk about programming systems in a way that can be shared and constructively debated rather than relying solely on personal impressions. Our framework is derived using a qualitative analysis of past programming systems. We outline two concrete ways of using our framework. First, we show how it can analyze a recently developed novel programming system. Then, we use it to identify an interesting unexplored point in the design space of programming systems. Much research effort focuses on building programming systems that are easier to use, accessible to non-experts, moldable and/or powerful, but such efforts are disconnected. They are informal, guided by the personal vision of their authors and thus are only evaluable and comparable on the basis of individual experience using them. By providing foundations for more systematic research, we can help programming systems researchers to stand, at last, on the shoulders of giants.
Understanding and measuring software engineer behavior: What can we learn from the behavioral sciences?

Allysson Allex Araújo,Marcos Kalinowski,Daniel Graziotin

2024-06-05

Abstract:This paper explores the intricate challenge of understanding and measuring software engineer behavior. More specifically, we revolve around a central question: How can we enhance our understanding of software engineer behavior? Grounded in the nuanced complexities addressed within Behavioral Software Engineering (BSE), we advocate for holistic methods that integrate quantitative measures, such as psychometric instruments, and qualitative data from diverse sources. Furthermore, we delve into the relevance of this challenge within national and international contexts, highlighting the increasing interest in understanding software engineer behavior. Real-world initiatives and academic endeavors are also examined to underscore the potential for advancing this research agenda and, consequently, refining software engineering practices based on behavioral aspects. Lastly, this paper addresses different ways to evaluate the progress of this challenge by leveraging methodological skills derived from behavioral sciences, ultimately contributing to a deeper understanding of software engineer behavior and software engineering practices.

Software Engineering
Reflecting on the Use of the Policy-Process-Product Theory in Empirical Software Engineering

Kelechi G. Kalu,Taylor R. Schorlemmer,Sophie Chen,Kyle Robinson,Erik Kocinare,James C. Davis

2023-08-24

Abstract:The primary theory of software engineering is that an organization's Policies and Processes influence the quality of its Products. We call this the PPP Theory. Although empirical software engineering research has grown common, it is unclear whether researchers are trying to evaluate the PPP Theory. To assess this, we analyzed half (33) of the empirical works published over the last two years in three prominent software engineering conferences. In this sample, 70% focus on policies/processes or products, not both. Only 33% provided measurements relating policy/process and products. We make four recommendations: (1) Use PPP Theory in study design; (2) Study feedback relationships; (3) Diversify the studied feedforward relationships; and (4) Disentangle policy and process. Let us remember that research results are in the context of, and with respect to, the relationship between software products, processes, and policies.

Software Engineering

It Will Never Work in Theory

It Will Never Work in Theory

Philosophical Smoke Signals: Theory and Practice in Information Systems Design

Theory-Software Translation: Research Challenges and Future Directions

An Experience Report On Applying Software Testing Academic Results In Industry: We Need Usable Automated Test Generation

Social Science Theories in Software Engineering Research

Improving software practice through education: Challenges and future trends

Industry-academia research collaboration and knowledge co-creation: Patterns and anti-patterns

Roundtable: The Future of Software Engineering for Internet Computing

Industry-Academia Research Collaboration in Software Engineering: The Certus Model

A Community's Perspective on the Status and Future of Peer Review in Software Engineering

Towards a Philosophy of Software Development: 40 Years after the Birth of Software Engineering

Four presumed gaps in the software engineering research community's knowledge

Software Citation in Theory and Practice

Software engineering and the SP Theory of Intelligence

Using grounded theory to study the experience of software development

The Four Pillars of Research Software Engineering

The Essence Theory of Software Engineering - Large-Scale Classroom Experiences from 450+ Software Engineering BSc Students

Technical Dimensions of Programming Systems

Understanding and measuring software engineer behavior: What can we learn from the behavioral sciences?

Reflecting on the Use of the Policy-Process-Product Theory in Empirical Software Engineering