We urgently need a culture of multi-operationalization in psychological research

Dino Carpentras

DOI: https://doi.org/10.1038/s44271-024-00084-7

2024-04-18

Abstract:Analysis of different operationalizations shows that many scientific results may be an artifact of the operationalization process. A culture of multi-operationalization may be needed for psychological research to develop valid knowledge.

What problem does this paper attempt to address?

Multicultural Counseling Competence: A Construct in Search of Operationalization

Charles R. Ridley,Debra Mollen,Katie Console,Caroliina Yin

DOI: https://doi.org/10.1177/0011000020988110

2021-04-12

The Counseling Psychologist

Abstract:The work of many great scholars has proliferated a sizable body of knowledge on the construct of multicultural counseling competence. However, the construct’s operationalization remains obscured, perplexing, and frustrating to practitioners who attempt to translate the scholarship into practice. We identify ten definitional problems that prevent the construct from evolving into a cohesive form that can inform practitioners’ work. These include: an indistinct purpose, culturally general/culturally specific divide, terminological interchange, confusing competency with competence, lack of integration, no definition, ambiguity, equivocation, circular reasoning, and divergence. Furthermore, the three major models of the construct—skills-based, adaptation, and process-oriented—share six limitations. They lack interdependence, prescriptive methods, deep incorporation of culture, coherent designs, conclusive research support, and they are oversimplifications. We call on the community of our fellow scholars to collaborate in reconceptualizing this complex construct into a sound, applicable guide for practitioners’ work with diverse clients.
The Importance of Separating Appropriateness into Impact and Feasibility for the Psychology of Creativity

Joel Chan,Christian D. Schunn

DOI: https://doi.org/10.1080/10400419.2023.2191919

IF: 2.032

2023-04-05

Creativity Research Journal

Abstract:Scientific progress on creativity research depends on having properly operationalized measures. In psychological research on creativity, it is common to operationalize creativity as the combination of novelty and appropriateness. However, the operationalization of appropriateness varies widely across researchers, studies, and domains (e.g. technical goodness, significance, elegance, usefulness, and feasibility). We argue that a core distinction between impact (how useful an idea is for solving the problem) and feasibility (how easy it is to realize the idea) underlies the variation. We further claim that this distinction is both possible to capture reliably in practice and psychologically significant. To test these claims, 318 ideas from 5 real-world social innovation problems (e.g. improving accessibility in elections) were rated for novelty, impact, and feasibility by a set of six experts selected for each of the 5 challenges. We find that all three constructs can be measured reliably and are statistically separable. Further, we show that distinguishing impact and feasibility reveals theoretically meaningful patterns of relationships with key psychological processes of creativity – analogy and conceptual combination – that would be difficult if impact and feasibility were conflated. These results demonstrate the theoretical importance of separating appropriateness into impact and feasibility for the psychology of creativity.

psychology, multidisciplinary, educational
Operationalizing Urie Bronfenbrenner’s Process-Person-Context-Time Model

Mu Xia,Xueliu Li,Jonathan R.H. Tudge

DOI: https://doi.org/10.1159/000507958

2020-01-01

Human Development

Abstract:Urie Bronfenbrenner spent much of his academic life designing and then refining what came to be known as the bioecological theory of human development. During this process, he came to a working operationalization of his theory – what he termed the process-person-context-time (PPCT) model. However, for all of the effort that he put into refining the theory, he provided very little information on how to apply the theory, or even the PPCT model, in research. In this paper we therefore suggest a way in which the theory can be operationalized more effectively, laying out the steps that we think should be followed. Specifically, these include (a) deciding on the developmental outcome of interest; (b) operationalizing the proximal processes most relevant to the outcome of interest; (c) operationalizing relevant person, context, and time factors, such that at least two contexts are included, in both contexts can be found persons who vary in their characteristics of interest, and data are gathered over time. Finally, and most importantly, the researcher must take seriously the fact that these interacting variables operate synergistically rather than independently. In this paper, our aim is to explain the importance of these four aspects and provide examples to help researchers who wish to use bioecological theory as its author intended. <br/>Human Development

psychology, developmental
Cultural Research in the Production and Operations Management Field

Richard Metters,Donna Marshall,Mark Pagell

DOI: https://doi.org/10.1561/0200000054

2019-01-01

Abstract:We summarize and categorize Operations Management (OM) research on two inter-related types of "culture": exogenous, or national culture and endogenous, or organizational culture. OM cultural research is far less than one percent of total OM research. We posit that of that small amount, much of OM cultural research is based on numerical approaches that have questionable validity. Qualitative work is highlighted. In addition to being a guide for research, this article is meant to provide substantive examples for teaching the importance of culture in OM.<h3>Suggested Citation</h3>Richard Metters, Donna Marshall and Mark Pagell (2019), "Cultural Research in the Production and Operations Management Field", Foundations and Trends® in Technology, Information and Operations Management: Vol. 13: No. 1-2, pp 1-150. http://dx.doi.org/10.1561/0200000054
People-Centric Operations: Achievements and Future Research Directions

Guillaume Roels,Bradley R. Staats

DOI: https://doi.org/10.2139/ssrn.3719847

2020-01-01

SSRN Electronic Journal

Abstract:As the nature of work has become more service-oriented, knowledge-intensive, and rapidly changing, people—be they workers or customers—have become more central to operational processes and have impacted operational outcomes in novel and perhaps more fundamental ways. Research in people-centric operations (PCO) studies how people affect the performance of operational processes. In this OM Forum, we define PCO as an area of study, offer a categorization scheme to take stock of where the field has allocated its attention to date, and offer our thoughts on promising directions for future research. The future of PCO is bright: Thanks to today’s availability of granular data, PCO researchers have numerous and growing opportunities to study, from both descriptive and prescriptive angles, the link between people’s behavior and operational performance.

English Else
Creativity and Risk Taking Aren't Rational: Behavioral Operations in MOT

Christoph H. Loch

DOI: https://doi.org/10.1111/poms.12666

IF: 4.638

2017-01-10

Production and Operations Management

Abstract:Behavioral Operations Management explicitly considers the effects of human behavior in process performance, influenced by cognitive biases, social preferences, and cultural norms. This broadening of Operations is even more critical in the context of the Management of Technology ( MOT ) than in the operations of established ongoing processes, because in innovation, people do not know well which tasks they will have to perform, they are exposed to risks, and they are subject to emergent interdependencies, all of which push psychological biases and social preferences to the fore. This article gives an overview of important behavioral challenges in MOT , setting them in the context of the phases of the stage gate process on the one hand, and the three levels of individual biases, group member interactions, and large group interactions (e.g., culture) on the other hand. The review suggests that previous work has not addressed a number of important questions in empirical effects and theoretical underpinnings, gaps that are very important for the performance of organizations in practice. The article concludes by offering opportunities for high‐impact future work.

operations research & management science,engineering, manufacturing
Replication and Reproducibility in Cross-Cultural Psychology

Taciano L. Milfont,Richard A. Klein

DOI: https://doi.org/10.1177/0022022117744892

2018-05-21

Abstract:Replication is the scientific gold standard that enables the confirmation of research findings. Concerns related to publication bias, flexibility in data analysis, and high-profile cases of academic misconduct have led to recent calls for more replication and systematic accumulation of scientific knowledge in psychological science. This renewed emphasis on replication may pose specific challenges to cross-cultural research due to inherent practical difficulties in emulating an original study in other cultural groups. The purpose of the present article is to discuss how the core concepts of this replication debate apply to cross-cultural psychology. Distinct to replications in cross-cultural research are examinations of bias and equivalence in manipulations and procedures, and that targeted research populations may differ in meaningful ways. We identify issues in current psychological research (analytic flexibility, low power) and possible solutions (preregistration, power analysis), and discuss ways to implement best practices in cross-cultural replication attempts.

psychology, social
SciOps: Achieving Productivity and Reliability in Data-Intensive Research

Erik C. Johnson,Thinh T. Nguyen,Benjamin K. Dichter,Frank Zappulla,Montgomery Kosma,Kabilar Gunalan,Yaroslav O. Halchenko,Shay Q. Neufeld,Kristen Ratan,Nicholas J. Edwards,Susanne Ressl,Sarah R. Heilbronner,Michael Schirner,Petra Ritter,Brock Wester,Satrajit Ghosh,Maryann E. Martone,Franco Pestilli,Dimitri Yatsenko

2024-11-07

Abstract:Scientists are increasingly leveraging advances in instruments, automation, and collaborative tools to scale up their experiments and research goals, leading to new bursts of discovery. Various scientific disciplines, including neuroscience, have adopted key technologies to enhance collaboration, reproducibility, and automation. Drawing inspiration from advancements in the software industry, we present a roadmap to enhance the reliability and scalability of scientific operations for diverse research teams tackling large and complex projects. We introduce a five-level Capability Maturity Model describing the principles of rigorous scientific operations in projects ranging from small-scale exploratory studies to large-scale, multi-disciplinary research endeavors. Achieving higher levels of operational maturity necessitates the adoption of new, technology-enabled methodologies, which we refer to as SciOps. This concept is derived from the DevOps methodologies that have revolutionized the software industry. SciOps involves digital research environments that seamlessly integrate computational, automation, and AI-driven efforts throughout the research cycle-from experimental design and data collection to analysis and dissemination, ultimately leading to closed-loop discovery. This maturity model offers a framework for assessing and improving operational practices in multidisciplinary research teams, guiding them towards greater efficiency and effectiveness in scientific inquiry.

Neurons and Cognition,Computers and Society
Operational Capabilities: The Secret Ingredient

Sarah Jinhui Wu,Steven A. Melnyk,Barbara B. Flynn

DOI: https://doi.org/10.1111/j.1540-5915.2010.00294.x

2010-11-01

Decision Sciences

Abstract:We develop a theoretical definition of operational capabilities, based on the strategic management and operations management literature, and differentiate this construct from the related constructs of resources and operational practices, drawing upon the resource‐based view of the firm as our foundation. We illustrate the key features of operational capabilities using the illustration of a restaurant kitchen. Because the traits of operational capabilities are distinct, they create a barrier to imitation, making them a potential source of competitive advantage. However, operational capabilities are particularly challenging to measure, because they emerge gradually and are tacit, embedded, and manifested differently across firms. In solving this measurement conundrum, we draw upon similar situations experienced by Schein (2004) and Eisenhardt and Martin (2000) in operationalizing organizational culture and dynamic capabilities. A taxonomy of six emergent operational capabilities is developed: operational improvement, operational innovation, operational customization, operational cooperation, operational responsiveness, and operational reconfiguration. A set of measurement scales is developed, in order to measure each of the operational capabilities, and validated using two different datasets. This allows replication of the psychometric properties of the multi‐item scales and helps to ensure the validity of the resulting measures.

management
Principles and Practices of Methodology and Methods in Cross-Cultural Psychology

Ype H. Poortinga,Johnny R. J. Fontaine

DOI: https://doi.org/10.1177/00220221221093811

2022-08-13

Abstract:Journal of Cross-Cultural Psychology, Volume 53, Issue 7-8, Page 847-859, August/September 2022. Principles of methodology in (cross-)cultural psychology are discussed and how these work out in practice. We propose that the frequently mentioned contrasts between context-specificity and universality of psychological functioning, and between qualitative and quantitative research traditions can be transcended by an empirical cycle in which both qualitative methods geared to exploration and quantitative methods geared to testing of hypotheses are acknowledged. We note issues in research due to non-random sampling, lack of psychometric equivalence of data, and nesting of individuals in populations. We argue that concerns about poor reproducibility in psychology cannot be ignored in cross-cultural psychology and make suggestions how research can be improved by treating this not as a threat but as an opportunity to expand cooperation.

psychology, social
Prespecification of Structure for Optimizing Data Collection and Research Transparency by Leveraging Conditional Independencies

Matthew J. Vowels

DOI: https://doi.org/10.48550/arXiv.2203.13331

2022-04-19

Abstract:Data collection and research methodology represents a critical part of the research pipeline. On the one hand, it is important that we collect data in a way that maximises the validity of what we are measuring, which may involve the use of long scales with many items. On the other hand, collecting a large number of items across multiple scales results in participant fatigue, and expensive and time consuming data collection. It is therefore important that we use the available resources optimally. In this work, we consider how a consideration for theory and the associated causal/structural model can help us to streamline data collection procedures by not wasting time collecting data for variables which are not causally critical for subsequent analysis. This not only saves time and enables us to redirect resources to attend to other variables which are more important, but also increases research transparency and the reliability of theory testing. In order to achieve this streamlined data collection, we leverage structural models, and Markov conditional independency structures implicit in these models to identify the substructures which are critical for answering a particular research question. In this work, we review the relevant concepts and present a number of didactic examples with the hope that psychologists can use these techniques to streamline their data collection process without invalidating the subsequent analysis. We provide a number of simulation results to demonstrate the limited analytical impact of this streamlining.

Methodology,Applications
Rating scales institutionalise a network of logical errors and conceptual problems in research practices: A rigorous analysis showing ways to tackle psychology's crises

Jana Uher

DOI: https://doi.org/10.3389/fpsyg.2022.1009893

2022-12-28

Abstract:This article explores in-depth the metatheoretical and methodological foundations on which rating scales-by their very conception, design and application-are built and traces their historical origins. It brings together independent lines of critique from different scholars and disciplines to map out the problem landscape, which centres on the failed distinction between psychology's study phenomena (e.g., experiences, everyday constructs) and the means of their exploration (e.g., terms, data, scientific constructs)-psychologists' cardinal error. Rigorous analyses reveal a dense network of 12 complexes of problematic concepts, misconceived assumptions and fallacies that support each other, making it difficult to be identified and recognised by those (unwittingly) relying on them (e.g., various forms of reductionism, logical errors of operationalism, constructification, naïve use of language, quantificationism, statisticism, result-based data generation, misconceived nomotheticism). Through the popularity of rating scales for efficient quantitative data generation, uncritically interpreted as psychological measurement, these problems have become institutionalised in a wide range of research practices and perpetuate psychology's crises (e.g., replication, confidence, validation, generalizability). The article provides an in-depth understanding that is needed to get to the root of these problems, which preclude not just measurement but also the scientific exploration of psychology's study phenomena and thus its development as a science. From each of the 12 problem complexes; specific theoretical concepts, methodologies and methods are derived as well as key directions of development. The analyses-based on three central axioms for transdisciplinary research on individuals, (1) complexity, (2) complementarity and (3) anthropogenicity-highlight that psychologists must (further) develop an explicit metatheory and unambiguous terminology as well as concepts and theories that conceive individuals as living beings, open self-organising systems with complementary phenomena and dynamic interrelations across their multi-layered systemic contexts-thus, theories not simply of elemental properties and structures but of processes, relations, dynamicity, subjectivity, emergence, catalysis and transformation. Philosophical and theoretical foundations of approaches suited for exploring these phenomena must be developed together with methods of data generation and methods of data analysis that are appropriately adapted to the peculiarities of psychologists' study phenomena (e.g., intra-individual variation, momentariness, contextuality). Psychology can profit greatly from its unique position at the intersection of many other disciplines and can learn from their advancements to develop research practices that are suited to tackle its crises holistically.
Learning from multi-paradigmatic sensitivity in cross-cultural management? Empirical and theoretical considerations

Sina Grosskopf,Christoph Barmeyer

DOI: https://doi.org/10.1177/14705958211019437

2021-06-03

International Journal of Cross Cultural Management

Abstract:Paradigms are basic assumptions about how social reality is perceived, understood and explained. Whereas most research is based on a single paradigm, few empirical papers show the advantages of using multiple paradigms within a study. This article pleads for multi-paradigm studies in cross-cultural management research in order to reach a more multifaceted representation of cultural phenomena. This is particularly consistent with the field of cross-cultural management, because it would be ethnocentric to consider intercultural situations only from one perspective, usually that of one’s own culture. The argument corresponds to the ambition of cross-cultural management to respect and adopt multiple (cultural) perspectives and, analogously, to achieve a ‘paradigmatic ethnorelativism’. Based on an intercultural situation, and therefore going beyond meta-theoretical reasoning, this article demonstrates multi-paradigmatic sensitivity in terms of the functionalist, interpretive and critical paradigms. The use of these theoretical concepts leads to multiple angles and a less ‘ethnocentric’ position, and hence to more nuanced knowledge creation with regard to the intercultural situation. The ‘blind spots’ of each paradigm, but also their complementarities, are discussed. Consequently, this article raises theoretical and practical implications for cross-cultural management by offering a way to a richer understanding of intercultural situations through openness to different paradigms.
Beyond single paradigms, pipelines, and outcomes: Embracing multiverse analyses in psychophysiology

Peter E. Clayson

DOI: https://doi.org/10.1016/j.ijpsycho.2024.112311

IF: 2.903

2024-01-30

International Journal of Psychophysiology

Abstract:Psychophysiological research is an inherently complex undertaking due to the nature of the data, and its analysis is characterized by many decision points that shape the final dataset and a study's findings. These decisions create a "multiverse" of possible outcomes, and each decision from study conceptualization to statistical analysis can lead to different results and interpretations. This review describes the concept of multiverse analyses, a methodological approach designed to understand the impact of different decisions on the robustness of the study's findings and interpretation. The emphasis is on transparently showcasing different reasonable approaches for constructing a final dataset and on highlighting the influence of various decision points, from experimental design to data processing and outcome selection. For example, the choice of an experimental task can significantly impact event-related brain potential (ERP) scores or skin conductance responses (SCRs), and different tasks might elicit unique variances in each measure. This review underscores the importance of transparently embracing the flexibility inherent in psychophysiological research and the potential consequences of not understanding the fragility or robustness of experimental findings. By navigating the intricate terrain of the psychophysiological multiverse, this review serves as an introduction, helping researchers to make informed decisions, improve the collective understanding of psychophysiological findings, and push the boundaries of the field.

psychology, experimental,physiology,neurosciences, biological
Polycultural psychology.

Michael W Morris,Chi-yue Chiu,Zhi Liu

DOI: https://doi.org/10.1146/annurev-psych-010814-015001

IF: 27.782

2014-01-01

Annual Review of Psychology

Abstract:We review limitations of the traditional paradigm for cultural research and propose an alternative framework, polyculturalism. Polyculturalism assumes that individuals' relationships to cultures are not categorical but rather are partial and plural; it also assumes that cultural traditions are not independent, sui generis lineages but rather are interacting systems. Individuals take influences from multiple cultures and thereby become conduits through which cultures can affect each other. Past literatures on the influence of multiple cultural identities and cultural knowledge legacies can be better understood within a polyculturalist rubric. Likewise, the concept elucidates how cultures are changed by contact with other cultures, enabling richer psychological theories of intercultural influence. Different scientific paradigms about culture imply different ideologies and policies; polyculturalism's implied policy of interculturalism provides a valuable complement to the traditional policy frames of multiculturalism and colorblindness.
Descriptive, Predictive and Explanatory Personality Research: Different Goals, Different Approaches, but a Shared Need to Move Beyond the Big Few Traits

René Mõttus,Dustin Wood,David M. Condon,Mitja D. Back,Anna Baumert,Giulio Costantini,Sacha Epskamp,Samuel Greiff,Wendy Johnson,Aaron Lukaszewski,Aja Murray,William Revelle,Aidan G.C. Wright,Tal Yarkoni,Matthias Ziegler,Johannes Zimmermann

DOI: https://doi.org/10.1002/per.2311

IF: 7

2020-12-20

European Journal of Personality

Abstract:We argue that it is useful to distinguish between three key goals of personality science—description, prediction and explanation—and that attaining them often requires different priorities and methodological approaches. We put forward specific recommendations such as publishing findings with minimum a priori aggregation and exploring the limits of predictive models without being constrained by parsimony and intuitiveness but instead maximizing out‐of‐sample predictive accuracy. We argue that naturally occurring variance in many decontextualized and multidetermined constructs that interest personality scientists may not have individual causes, at least as this term is generally understood and in ways that are human‐interpretable, never mind intervenable. If so, useful explanations are narratives that summarize many pieces of descriptive findings rather than models that target individual cause–effect associations. By meticulously studying specific and contextualized behaviours, thoughts, feelings and goals, however, individual causes of variance may ultimately be identifiable, although such causal explanations will likely be far more complex, phenomenon‐specific and person‐specific than anticipated thus far. Progress in all three areas—description, prediction and explanation—requires higher dimensional models than the currently dominant 'Big Few' and supplementing subjective trait‐ratings with alternative sources of information such as informant‐reports and behavioural measurements. Developing a new generation of psychometric tools thus provides many immediate research opportunities. © 2020 European Association of Personality Psychology

psychology, social
Don't let perfect be the enemy of better: In defense of unparameterized megastudies

Wei Li,Joshua K. Hartshorne

DOI: https://doi.org/10.1017/s0140525x23002121

2024-01-01

Behavioral and Brain Sciences

Abstract:Abstract The target article argues researchers should be more ambitious, designing studies that systematically and comprehensively explore the space of possible experiments in one fell swoop. We argue that while “systematic” is rarely achievable, “comprehensive” is often enough. Critically, the recent popularization of massive online experiments shows that comprehensive studies are achievable for most cognitive and behavioral research questions.

behavioral sciences,neurosciences,psychology, biological
Phenomena complexity, disciplinary consensus, and experimental versus correlational research in psychological science

Dean Keith Simonton

DOI: https://doi.org/10.1017/s0140525x23002339

2024-01-01

Behavioral and Brain Sciences

Abstract:Abstract The target article ignores the crucial role of correlational methods in the behavioral and social sciences. Yet such methods are often mandated by the greater complexity of the phenomena investigated. This necessity is especially conspicuous in psychological research where its position in the hierarchy of the sciences implies the need for both experimental and correlational investigations, each featuring distinct assets.

behavioral sciences,neurosciences,psychology, biological
Theory building for empirical software engineering in qualitative research: Operationalization

Jorge Pérez,Jessica Díaz,Ángel González-Prieto,Sergio Gil-Borrás

2024-12-03

Abstract:Context: This work is part of a research project whose ultimate goal is to systematize theory building in qualitative research in the field of software engineering. The proposed methodology involves four phases: conceptualization, operationalization, testing, and application. In previous work, we performed the conceptualization of a theory that investigates the structure of IT departments and teams when software-intensive organizations adopt a culture called DevOps. Objective: This paper presents a set of procedures to systematize the operationalization phase in theory building and their application in the context of DevOps team structures. Method: We operationalize the concepts and propositions that make up our theory to generate constructs and empirically testable hypotheses. Instead of using causal relations to operationalize the propositions, we adopt logical implication, which avoids the problems associated with causal reasoning. Strategies are proposed to ensure that the resulting theory aligns with the criterion of parsimony. Results: The operationalization phase is described from three perspectives: specification, implementation, and practical application. First, the operationalization process is formally defined. Second, a set of procedures for operating both concepts and propositions is described. Finally, the usefulness of the proposed procedures is demonstrated in a case study. Conclusions: This paper is a pioneering contribution in offering comprehensive guidelines for theory operationalization using logical implication. By following established procedures and using concrete examples, researchers can better ensure the success of their theory-building efforts through careful operationalization.

Software Engineering,Logic in Computer Science
Pro-Environmental Organizational Culture: Its Essence and a Concept for Its Operationalization

Katarzyna Piwowar-Sulej

DOI: https://doi.org/10.3390/su12104197

IF: 3.9

2020-05-20

Sustainability

Abstract:Organizational culture is currently considered a critical factor in achieving success in any business activity, including the implementation of the idea of sustainability. Sustainability-driven organizations incorporate a triple bottom line approach, which promotes the collateral achievement of companies’ environmental, economic, and social goals. The first bottom line is related to environmental sustainability. This bottom line should be facilitated by a pro-environmental organizational culture (PE culture). A company should strive to bring this culture to the highest possible level. Although in the literature on the subject some research on the factors that influence employees’ pro-environmental behavior have been presented, environmental sustainability in the context of organizational culture has received comparatively little attention. So far, the concept of levels of PE culture and the corresponding measurement instrument have not been developed. The aim of this paper is to present the concept of a pro-environmental organizational culture and its operationalization to support the achievement of sustainability goals. Operationalization is the process of determining the extension of a concept, which in turn transforms the concept into a metric. This paper presents a four-level concept of PE culture and a research instrument developed for the purpose of measuring the cultural level in organizations. The instrument was tested in a manufacturing company.

environmental sciences,environmental studies,green & sustainable science & technology

We urgently need a culture of multi-operationalization in psychological research

Multicultural Counseling Competence: A Construct in Search of Operationalization

The Importance of Separating Appropriateness into Impact and Feasibility for the Psychology of Creativity

Operationalizing Urie Bronfenbrenner’s Process-Person-Context-Time Model

Cultural Research in the Production and Operations Management Field

People-Centric Operations: Achievements and Future Research Directions

Creativity and Risk Taking Aren't Rational: Behavioral Operations in MOT

Replication and Reproducibility in Cross-Cultural Psychology

SciOps: Achieving Productivity and Reliability in Data-Intensive Research

Operational Capabilities: The Secret Ingredient

Principles and Practices of Methodology and Methods in Cross-Cultural Psychology

Prespecification of Structure for Optimizing Data Collection and Research Transparency by Leveraging Conditional Independencies

Rating scales institutionalise a network of logical errors and conceptual problems in research practices: A rigorous analysis showing ways to tackle psychology's crises

Learning from multi-paradigmatic sensitivity in cross-cultural management? Empirical and theoretical considerations

Beyond single paradigms, pipelines, and outcomes: Embracing multiverse analyses in psychophysiology

Polycultural psychology.

Descriptive, Predictive and Explanatory Personality Research: Different Goals, Different Approaches, but a Shared Need to Move Beyond the Big Few Traits

Don't let perfect be the enemy of better: In defense of unparameterized megastudies

Phenomena complexity, disciplinary consensus, and experimental versus correlational research in psychological science

Theory building for empirical software engineering in qualitative research: Operationalization

Pro-Environmental Organizational Culture: Its Essence and a Concept for Its Operationalization