The Semiotic Machine

Eric Engle

DOI: https://doi.org/10.48550/arXiv.0809.0461

2008-09-03

Abstract:A semiotic model of the user interface in human-computer interaction. Algorithmic sign, semotics, algorithmic art.

Human-Computer Interaction

What problem does this paper attempt to address?

Design Semiotics Based Icon Design

Zhou Yuxiao,Luo Shijian,Chen Gencai

DOI: https://doi.org/10.3969/j.issn.1003-9775.2012.10.009

2012-01-01

Journal of Computer-Aided Design & Computer Graphics

Abstract:An approach of design semiotics based icon design was proposed for exploring application of icon recognition in design practice.The icon was divided into four dimensions including semantic dimension,syntactic dimension,context dimension and pragmatics dimension,in which the innovative cognition and integration of icon design process and user study were discussed,a closed icon design theory was developed.After the analysis of design semiotics based icon design process and icon algebra theory,a design semiotics based icon design expression model was presented;the importance of semantic and syntactic dimensions was put forward.With.NET technology,a design semiotics based icon design computer system IDAS1.0 was set up on Windows 7 platform,including rapid icon design component,design semiotics evaluation component and knowledge base component.Based on the IDAS1.0,icon design can be accomplished rapidly,the knowledge base can be built,and the expert design semiotics evaluation can be achieved.Thus the design semiotics based icon design theory was tested.
The semiotics of medical image Segmentation

John S H Baxter,Eli Gibson,Roy Eagleson,Terry M Peters

DOI: https://doi.org/10.1016/j.media.2017.11.007

Abstract:As the interaction between clinicians and computational processes increases in complexity, more nuanced mechanisms are required to describe how their communication is mediated. Medical image segmentation in particular affords a large number of distinct loci for interaction which can act on a deep, knowledge-driven level which complicates the naive interpretation of the computer as a symbol processing machine. Using the perspective of the computer as dialogue partner, we can motivate the semiotic understanding of medical image segmentation. Taking advantage of Peircean semiotic traditions and new philosophical inquiry into the structure and quality of metaphors, we can construct a unified framework for the interpretation of medical image segmentation as a sign exchange in which each sign acts as an interface metaphor. This allows for a notion of finite semiosis, described through a schematic medium, that can rigorously describe how clinicians and computers interpret the signs mediating their interaction. Altogether, this framework provides a unified approach to the understanding and development of medical image segmentation interfaces.
Interfacing the human/machine

Benjamin Lipp,Sascha Dickel

DOI: https://doi.org/10.1080/1600910X.2021.2012709

2022-01-25

Distinktion

Abstract:Contemporary discourse on information and communication technology suggests that humans and machines are increasingly converging. However, in this article, we argue that for analysts of digital society it is necessary to understand the simultaneity with which humans and machines are both interconnected and separated from one another. Here, we propose to follow and trace human/machine interfacing , i.e. the manifold practices, by which humans and machines become interconnected, by being kept apart. Drawing on theoretical resources from feminist science and technology studies and the philosophy of technology, we extend common 'objective' notions of the interface and propose a performative and ecological framework for human-machine interfacing. We illustrate this framework with two contrasting cases, embodied social robotics and communicative software bots. Social robots denote a rather fragile technology, showing the precariousness of efforts to engender phenomena of human-machine communication. By contrast, communicative software bots can rely on vast digital infrastructures, which create the impression of symmetry between humans and machines by rendering ontological differences invisible.
The Enigmatics of Affect

Phoebe Sengers,Rainer Liesendahi,Werner Magar,Christoph Seibert,Boris Müller,Thorsten Joachims,Weidong Geng,Pia Mårtensson,Kristina Höök

DOI: https://doi.org/10.1145/778712.778728

2002-01-01

Abstract:Affective computation generally focuses on the informatics of affect: structuring, formalizing, and representing emotion as informational units. We propose instead an enigmatics of affect, a critical technical practice that respects the rich and undefinable complexities of human affective experience. Our interactive installation, the Influencing Machine, allows users to explore a dynamic landscape of emotionally expressive sound and child-like drawings, using a tangible, intuitive input device that supports open-ended engagement. The Influencing Machine bridges the subjective experience of the user and the necessary objective rationality of the underlying code. It functions as a cultural probe, reflecting and challenging users to reflect on the cultural meaning of affective computation.
The Semiotics of Cyberculture: the Example of Artistic Discourse

Ewa Szczęsna

DOI: https://doi.org/10.35757/kis.2019.63.3.3

2019-09-27

Kultura i Społeczeństwo

Abstract:In this text the author reflects on the semiotic existence of culture discourses in connection with the rapid development of digital technology. The author analyzes selected texts of digital art as examples of the transformation in how works of literature, sculpture, or film exist. The article covers how movable font, which changes in shape and color, participates in shaping literary meanings; the creation of semiotic and interactive figures; the textualization of the user’s actions and body; dematerialization; processuality; narrativization; the temporalizing of sculpture, which changes before the eyes of the recipient; the presence of alternative narratives in literature and film; multi-variant plots co-created by the recipient; and the artistic use of other discourses. The author proves that the structure and specificity of the digital sign lies at the base of the changes. The digital sign is immaterial, programmable, and hybrid, and combines aspects of expression, meaning, and action, which makes it efficient.
Human-Machine Interaction in the Light of Turing and Wittgenstein

Charles Bodon

2023-08-07

Abstract:We propose a study of the constitution of meaning in human-computer interaction based on Turing and Wittgenstein's definitions of thought, understanding, and decision. We show by the comparative analysis of the conceptual similarities and differences between the two authors that the common sense between humans and machines is co-constituted in and from action and that it is precisely in this co-constitution that lies the social value of their interaction. This involves problematizing human-machine interaction around the question of what it means to "follow a rule" to define and distinguish the interpretative modes and decision-making behaviors of each. We conclude that the mutualization of signs that takes place through the human-machine dialogue is at the foundation of the constitution of a computerized society.

General Literature
Signs and Supersigns in Deep Learning

Razvan Andonie,Bogdan Musat

DOI: https://doi.org/10.15837/ijccc.2024.1.6392

IF: 2.635

2024-01-04

International Journal of Computers Communications & Control

Abstract:Semiotics is the study of signs and sign-using behavior. Computational semiotics is an interdisciplinary field which proposes a new kind of approach to intelligent systems, where an explicit account for the notion of sign is prominent. Our fundamental thesis is that information concentration processes appear in successive layers of deep learning models: each layer aggregates information from the previous layer of the network. In computational semiotics, this information concentration is known as superization, and it is accompanied by a decrease of entropy: signs are aggregated into supersign. Our interdisciplinary approach enables us to depict superization processes within deep learning models. This is a novel semantic interpretation of deep learning. We use concepts from computational semiotics to explain decision processes in deep learning. Semiotic tools can be used to optimize the architecture of deep neural networks. Interpretability/explainability and architecture optimization of neural models are currently among the hottest topics in machine learning. We illustrate our semiotic approach with several applications. Our contribution can be seen as the initial move in establishing a cohesive semiotic framework for deep learning models.

computer science, information systems,automation & control systems
The Interplay of Signs and Visuals: Unveiling the Symbiotic Relationship Between Semiotics and Visual Communication

Alard Travere

DOI: https://doi.org/10.56397/jlcs.2023.09.05

2023-09-01

Abstract:This article explores the symbiotic relationship between semiotics and visual communication. Semiotics provides the framework for understanding the meanings and interpretations of signs and symbols within visuals, while visuals act as signs themselves, conveying messages and eliciting responses from viewers. The article discusses how semiotics enhances visual communication by guiding the selection and arrangement of visual elements, considering cultural codes and conventions, and leveraging symbolism and metaphor. It also explores emerging trends in semiotics and visual communication, the potential of technology in enhancing this relationship, and ethical considerations in visual communication. By understanding and embracing this symbiotic relationship, visual communicators can create impactful and ethical visual messages that effectively engage audiences.

Linguistics,Art,Computer Science
A category theory approach to the semiotics of machine learning

Gangle, Rocco,Caterina, Gianluca

DOI: https://doi.org/10.1007/s10472-024-09932-y

IF: 1.019

2024-02-10

Annals of Mathematics and Artificial Intelligence

Abstract:The successes of Machine Learning, and in particular of Deep Learning systems, have led to a reformulation of the Artificial Intelligence agenda. One of the pressing issues in the field is the extraction of knowledge out of the behavior of those systems. In this paper we propose a semiotic analysis of that behavior, based on the formal model of learners . We analyze the topos-theoretic properties that ensure the logical expressivity of the knowledge embodied by learners. Furthermore, we show that there exists an ideal universal learner , able to interpret the knowledge gained about any possible function as well as about itself, which can be monotonically approximated by networks of increasing size.

computer science, artificial intelligence,mathematics, applied
Semiotics of Communication: From Semiosis of Nature to Culture

Irene Machado,Vinícius Romanini

DOI: https://doi.org/10.1007/s12304-011-9117-1

2011-04-15

Biosemiotics

Abstract:Communication Studies currently undergoes a crisis of paradigms that requires an ontological review that must begin with a debate about the conditions of possibility of every communicational phenomena. In this article we argue that semiosis offers a conceptual framework that allows for the study of communication as qualitative action. Semiosis, or the action of the sign, is here defined as a fundamental process based on perception that models the world of species, creating cognition and culture. At the core of semiosis are dynamic structures that the authors have defined as ‘ontological diagrams’. The first purpose of Semiotics of Communication is to understand how these modeling systems evolve ontologically and phylogenically, producing, in the case of human culture, means of communication ever more varied and technologically advanced.

history & philosophy of science
Algorithmic Subjectivities

Eric P. S. Baumer,Alex S. Taylor,Jed R. Brubaker,Micki McGee

DOI: https://doi.org/10.1145/3660344

IF: 4.106

2024-04-27

ACM Transactions on Computer-Human Interaction

Abstract:This paper considers how subjectivities are enlivened in algorithmic systems. We first review related literature to clarify how we see “subjectivities” as emerging through a tangled web of processes and actors. We then offer two case studies exemplifying the emergence of algorithmic subjectivities: one involving computational topic modeling of blogs written by parents with children on the autism spectrum, and one involving algorithmic moderation of social media content. Drawing on these case studies, we then articulate a series of qualities that characterizes algorithmic subjectivities. We also compare and contrast these qualities with a number of related concepts from prior literature to articulate how algorithmic subjectivities constitutes a novel theoretical contribution, as well as how it offers a focal lens for future empirical investigation and for design. In short, this paper points out how certain worlds are being made and/or being made possible via algorithmic systems, and it asks HCI to consider what other worlds might be possible.

computer science, information systems, cybernetics
Introduction: A Sociosemiotic Exploration of Identity and Discourse

Le Cheng,Ning Ye,David Machin

DOI: https://doi.org/10.1515/sem-2020-0030

2020-01-01

Semiotica

Abstract:Among the categories of the telecom and internet frauds, the online romance scam is of particular concern for its sharp rise of victim numbers and the huge amount of cost. A social semiotic approach could be used to investigate the victim identity of the online romance scam from the aspects of the (re)construction and interpretation of discursive practices. The range of papers in this section shows that the study of text, context and the way that people use semiotic resources to produce communication, to create and manage events and to interpret them.
Living Machines: Metaphors We Live By

Nora S. Vaage

DOI: https://doi.org/10.1007/s11569-019-00355-2

2019-12-10

NanoEthics

Abstract:Abstract Within biology and in society, living creatures have long been described using metaphors of machinery and computation: ‘bioengineering’, ‘genes as code’ or ‘biological chassis’. This paper builds on Lakoff and Johnson’s (1980) argument that such language mechanisms shape how we understand the world. I argue that the living machines metaphor builds upon a certain perception of life entailing an idea of radical human control of the living world, looking back at the historical preconditions for this metaphor. I discuss how design is perceived to enable us to shape natural beings to our will, and consider ethical, epistemological and ontological implications of the prevalence of this metaphor, focusing on its use within synthetic biology. I argue that we urgently need counter-images to the dominant metaphor of living machines and its implied control and propose that artworks can provide such counter-images through upsetting the perception of life as controllable. This is argued through discussion of artworks by Oron Catts and Ionat Zurr, by Tarsh Bates and by Ai Hasegawa, which in different ways challenge mechanistic assumptions through open-ended engagement with the strangeness and messiness of life.

ethics,history & philosophy of science
Análise semiótica de videogames: uma aposta na interdisciplinaridade

J. Angelo

2017-03-06

Abstract:This research aims to bring together the French Semiotics theory to the videogame studies, through an interdisciplinary model of analysis. We consider the videogame an updated paradigm in contemporary interactions, with the power to influence different social strata what can even be used to reinforce ideologies and to influence opinions. The interdisciplinary approach is characterized by the application of the semiotic methodology, with concepts as "syncretism" and "degrees of intimacy", presented by Hjelmslev, the tensive approach, in accordance to the “intensive/extensive" model proposed by Zilberberg, and the videogame studies, systematized by Jesper Juul. We demonstrate links between basic, narrative and discursive levels and the concept of "rules". We also discuss the concepts of implication/concession in relation to the classification of videogames "real" and "fictional" components. At last, we investigate how such trends are articulated to maximize or to minimize sensitive and intelligible components of the videogame, which are responsible to ensure the engagement and loyalty of the player.

Psychology,Sociology,Linguistics,Computer Science
Syntropic Counterpoints

Predrag K. Nikolic,Ruiyang Liu,Shengcheng Luo

DOI: https://doi.org/10.1145/3474085.3478541

2021-01-01

Abstract:In the artwork Syntropic Counterpoints: Metaphysics of The Machines, we tend to explore phenomena of AI aesthetic and challenge machine abstraction. Our approach toward the liberation of machine creativity is through the use of words and grammar as a creative tool humans developed to express worlds "beyond" the world, existing and non-existing realities. We are lead by Nietzsche's claim that grammar is the "Metaphysics of the People," as such grammar, content, and vision generated during the philosophical discussion between our AI clones is "Metaphysics of Machines" through we can experience their realities and start to question our own.
The semiotic functioning of synthetic media

Auli Viidalepp

DOI: https://doi.org/10.22503/inftars.xxii.2022.4.9

2022-12-01

Információs Társadalom

Abstract:The interpretation of many texts in the everyday world is concerned with their truth value in relation to the reality around us. The recent publication experiments with computer-generated texts have shown that the distinction between true and false, or reality and fiction, is not always clear from the text itself. Essentially, in today’s media space, one may encounter texts, videos or images that deceive the reader by displaying nonsensical content or nonexistent events, while nevertheless appearing as genuine human-produced messages. This article outlines certain problems with artificial intelligence (AI)-generated content, and frames the issue as a problem of recognising its proper referential reality. Examples include generative texts, deepfakes and their functioning in contemporary culture. The article makes use of the concepts of mimicry and nonsense to reveal the elements and counterparts in the communicative processes involving generated media.

information science & library science
Machinic Surrogates: Human-Machine Relationships in Computational Creativity

Ardavan Bidgoli,Eunsu Kang,Daniel Cardoso Llach

DOI: https://doi.org/10.48550/arXiv.1908.01133

2019-08-03

Abstract:Recent advancements in artificial intelligence (AI) and its sub-branch machine learning (ML) promise machines that go beyond the boundaries of automation and behave autonomously. Applications of these machines in creative practices such as art and design entail relationships between users and machines that have been described as a form of collaboration or co-creation between computational and human agents. This paper uses examples from art and design to argue that this frame is incomplete as it fails to acknowledge the socio-technical nature of AI systems, and the different human agencies involved in their design, implementation, and operation. Situating applications of AI-enabled tools in creative practices in a spectrum between automation and autonomy, this paper distinguishes different kinds of human engagement elicited by systems deemed automated or autonomous. Reviewing models of artistic collaboration during the late 20th century, it suggests that collaboration is at the core of these artistic practices. We build upon the growing literature of machine learning and art to look for the human agencies inscribed in works of computational creativity, and expand the co-creation frame to incorporate emerging forms of human-human collaboration mediated through technical artifacts such as algorithms and data.

Human-Computer Interaction,Machine Learning
Communication of Social Agents and the Digital City - A Semiotic Perspective

Victor V. Kryssanov,Masayuki Okabe,Koh Kakusho,Michihiko Minoh

DOI: https://doi.org/10.48550/arXiv.cs/0605121

IF: 14.4

2006-05-25

Artificial Intelligence

Abstract:This paper investigates the concept of digital city. First, a functional analysis of a digital city is made in the light of the modern study of urbanism; similarities between the virtual and urban constructions are pointed out. Next, a semiotic perspective on the subject matter is elaborated, and a terminological basis is introduced to treat a digital city as a self-organizing meaning-producing system intended to support social or spatial navigation. An explicit definition of a digital city is formulated. Finally, the proposed approach is discussed, conclusions are given, and future work is outlined.
A Biosemiotic Approach to the Problem of Structure and Agency

Shahram Rafieian

DOI: https://doi.org/10.1007/s12304-011-9121-5

2011-05-21

Biosemiotics

Abstract:A human being is the simultaneous composite of several different levels of being, from atomic and subatomic to the level of complex social interaction, and these levels are nested within the individual hierarchically (lower levels giving rise to higher levels, etc.). One of the most important and influential approaches developed in the history of science has been that of systems theory and systemic thinking, in which the different levels of the hierarchy, and the interactions between those levels, are considered simultaneously. Although this model provides a comprehensive view of biological being, the transition from one level to the other is not well defined in it. Uexküll and Pauli (Advances: Journal of the Institute for 417 the Advancement of Health 3:158–174, 1986) suggested that semiosis is the translator of the events from one level to the other. From a psychological point of view, a myriad of semiotic events happen inside an individual, and it has been suggested that among other semiotic events, inner speech plays an important role in mediating personal agency. Dialogical theories of the self, Jungian psychology and hypnosis research evidence show that there is a semiotic multiplicity in human agency and consciousness, and that these multiple streams are all converge to a central semiotic singularity. I argue in this paper that by taking a biosemiotic point of view, human ‘agency’ may be defined as the ability of an individual to direct the incoming and internal streams of semioses and the ability to create an integrative and superordinate new stream of semiosis in addition to the upwardly and downwardly component ones, and how such a view might open a new door for research into the concept of human ‘personality’ and ‘agency’.

history & philosophy of science

The Semiotic Machine

Design Semiotics Based Icon Design

The semiotics of medical image Segmentation

Interfacing the human/machine

The Enigmatics of Affect

The Semiotics of Cyberculture: the Example of Artistic Discourse

Human-Machine Interaction in the Light of Turing and Wittgenstein

Signs and Supersigns in Deep Learning

The Interplay of Signs and Visuals: Unveiling the Symbiotic Relationship Between Semiotics and Visual Communication

A category theory approach to the semiotics of machine learning

Semiotics of Communication: From Semiosis of Nature to Culture

Algorithmic Subjectivities

Introduction: A Sociosemiotic Exploration of Identity and Discourse

Living Machines: Metaphors We Live By

Análise semiótica de videogames: uma aposta na interdisciplinaridade

Syntropic Counterpoints

The semiotic functioning of synthetic media

Machinic Surrogates: Human-Machine Relationships in Computational Creativity

Communication of Social Agents and the Digital City - A Semiotic Perspective

A Biosemiotic Approach to the Problem of Structure and Agency