The grand challenge: facing up to proteomics.

J. Wooley

DOI: https://doi.org/10.1016/S0167-7799(02)01996-0

2002-08-01

Trends in Biotechnology

Abstract:

What problem does this paper attempt to address?

Imbalanced Data Classification Algorithm Based on Integrated Sampling and Ensemble Learning.

Yan Han,Mingxiang He,Qixian Lu

DOI: https://doi.org/10.1007/978-981-13-5841-8_64

2018-01-01

Abstract:In order to alleviate the impact of imbalanced data on support vector machine (SVM), an integrated hybrid sampling imbalanced data classification method is proposed. First, the imbalance rate of imbalanced data is reduced by the ADASYN-NCL (Adaptive Synthetic Sampling Technique—Domain Cleanup Rule Downsampling Method) hybrid sampling method. Then, the AdaBoost algorithm framework is used to give different weight adjustments to the misclassification of minority and majority classes, and selectively integrate several classifiers to obtain better classification. Finally, use the 10 sets of imbalanced data in the KEEL database as test objects, and F-value and G-mean are used as evaluation indicators to verify the performance of the classification algorithm. The experimental results show that the classification algorithm has certain advantages for the classification effect of imbalanced data sets.
Hybrid SVM algorithm oriented to classifying imbalanced datasets

Dongqi Liu,Zhijian Chen,Yin Xu,Feiteng Li

DOI: https://doi.org/10.3969/j.issn.1001-3695.2018.04.014

2018-01-01

Abstract:In order to improve the classification accuracy of traditional support vector machine (SVM) for imbalanced datasets,solving the problem that classifier had a low performance on minority class,this paper proposed a hybrid SVM algorithm.It combined adaptive synthetic sampling(ADASYN) algorithm with different error cost(DEC) algorithm to improve the bias of hyperplane caused by imbalanced datasets,and then it introduced a new correction algorithm to prediction model so as to improve the prediction model's adaptability to different data characteristics.It tested the proposed algorithm on 7 sets of realworld imbalanced datasets from UCI database.The experiment result shows that the hybrid SVM algorithm is able to surpass or match the state-of-the-art algorithms on each dataset,and it increases the classification performance by an average of 2.0％ to 20.9％.It shows that the proposed algorithm is effective and robust.
The Balanced Multi-Modal Spiking Neural Networks with Online Loss Adjustment and Time Alignment

Jianing Han,Jiangrong Shen,Qi Xu,Jian Liu,Huajin Tang

DOI: https://doi.org/10.1109/icme57554.2024.10687515

2024-01-01

Abstract:Optimizing multi-modal learning of SNNs has the advantages of energy efficiency and performance improvements. However, modality imbalance in multi-modal SNNs results in performance decline due to heterogeneity of modalities and temporal inconsistencies across different SNNs branches. In this paper, we propose the Balanced Multi-modal SNNs (BM-SNNs) model, equipped with a novel online loss adjustment (LA) algorithm and time alignment (TA) modules, ultimately achieving balanced training across multiple modalities. LA supervises the learning of uni-modal feature extractors by adding unimodal loss components without additional classifier. Moreover, the modulation factors enable the adaptive adjustment of unimodal learning rate. Furthermore, the proposed TA adopts the optimal timestep for different modalities to avoid information redundancy. Experimental results reveal that BM-SNNs model improves the performance of both multi-modal model and unimodal branches through adaptively exploiting intra-modal and cross-modal information.
CNUSVM: Hybrid CNN-Uneven SVM Model for Imbalanced Visual Learning

Geng Mengyue,Wang Yaowei,Tian Yonghong,Huang Tiejun

DOI: https://doi.org/10.1109/BigMM.2016.19

2016-01-01

Abstract:Recently, deep Convolutional Neural Networks (CNNs) have been used to achieve state-of-the-art performance on a wide range of visual learning tasks. However, when facing some imbalanced learning tasks where the training samples are unevenly distributed among different classes, CNNs tend to produce performance bias toward the majority class, making them not suitable for applications in which the recognition ability on the minority class is highly valued. To address the problem, this paper proposes a hybrid classification model by combining CNN with Support Vector Machine (SVM) that has uneven margins. In this model, CNN works as a feature extractor and the extracted features are then sent into a L2-SVM with linear uneven margins. We also develop a gradient-descent learning approach for this hybrid CNN-uneven SVM (CNUSVM) model by minimizing an uneven margin based L2-hinge loss. Our experiments on two benchmark datasets show that the CNUSVM model can make more favorable decisions for imbalanced visual learning tasks in comparison with the standard CNN and the hybrid CNN-SVM model.
CMCI: A Robust Multimodal Fusion Method for Spiking Neural Networks

Runhao Jiang,Jianing Han,Yingying Xue,Ping Wang,Huajin Tang

DOI: https://doi.org/10.1007/978-981-99-8067-3_12

2024-01-01

Abstract:Human understand the external world through a variety of perceptual processes such as sight, sound, touch and smell. Simulating such biological multi-sensory fusion decisions using a computational model is important for both computer and neuroscience research. Spiking Neural Networks (SNNs) mimic the neural dynamics of the brain, which are expected to reveal the biological multimodal perception mechanism. However, existing works of multimodal SNNs are still limited, and most of them only focus on audiovisual fusion and lack systematic comparison of the performance and robustness of the models. In this paper, we propose a novel fusion module called Cross-modality Current Integration (CMCI) for multimodal SNNs and systematically compare it with other fusion methods on visual, auditory and olfactory fusion recognition tasks. Besides, a regularization technique called Modality-wise Dropout (ModDrop) is introduced to further improve the robustness of multimodal SNNs in missing modalities. Experimental results show that our method exhibits superiority in both modality-complete and missing conditions without any additional networks or parameters.
Self-adaptive cost weights-based support vector machine cost-sensitive ensemble for imbalanced data classification

Xinmin Tao,Qing Li,Wenjie Guo,Chao Ren,Chenxi Li,Rui Liu,Junrong Zou

DOI: https://doi.org/10.1016/j.ins.2019.02.062

IF: 8.1

2019-06-01

Information Sciences

Abstract:Imbalanced data classification poses a major challenge in data mining community. Although standard support vector machine can generally show relatively robust performance in dealing with the classification problems of imbalanced data set, it is a typical overall accuracy-oriented algorithm which results in the final decision boundary biasing toward the majority class. Some ensemble methods have emerged as meta-techniques for improving the generalization performance of existing learning algorithms. In this paper, we propose a novel self-adaptive cost weights-based support vector machine cost-sensitive ensemble for imbalanced data classification. In the proposed approach, to guarantee the consistency of optimization objectives between weak learners and boosting scheme, we not only apply cost-sensitive SVMs as basic weak leaner but also simultaneously modify the standard boosting scheme to cost-sensitive ones. In order to ensure more training minority instances for successive classifiers, especially borderline minority instances, we also present a self-adaptive sequential misclassification cost weights determination method. The method can self-adaptively consider the different contribution of minority instances to the form of SVM classifiers at each iteration based on the preceding obtained classifier during boosting, which can allow it to produce diverse classifiers and thus improve its generalization performance. In the experiments, we analyze and discuss the effect of different parameters on the performance and some suggestions are also provided. The extensive experimental results on the different imbalanced datasets demonstrate that the proposed approach can achieve better generalization performance in terms of G-Mean and F-Measure as compared to the other existing imbalanced dataset classification techniques.

computer science, information systems
Adaptive learning cost‐sensitive convolutional neural network

Yun Hou,Hong Fan,Li Li,Bailin Li

DOI: https://doi.org/10.1049/cvi2.12027

IF: 1.484

2021-03-22

IET Computer Vision

Abstract:Real‐world classification often encounters a problem called class imbalance. When the data of some classes are redundant than that of other classes, traditional classifiers usually bias their decision boundaries to the redundant majority classes. Most proposed cost‐sensitive strategies often ignore the hard‐to‐learn examples or have a large amount of hyper‐parameters. This article proposes an adaptive learning cost‐sensitive convolutional neural network to solve this problem. During the training process, the proposed method embeds a class‐dependent cost to each class in the global error, making the decision boundary bias to the minority classes. Meanwhile, a distribution weight is assigned to each example to enhance the learning of the hard‐to‐learn examples. Both the class‐dependent costs and distribution weights are learnt automatically in the net. This cost‐sensitive approach makes the algorithm focus on the examples in the minority classes as well as the hard‐to‐learn examples in each class. Besides, this approach can be applied to both binary and multi‐class image classification problems without any modification. Experiments are conducted on four image classification datasets to evaluate this algorithm. The experimental results show that the proposed method achieves better performance than the baseline algorithms and some other algorithms.
Modern synergetic neural network for imbalanced small data classification

Zihao Wang,Haifeng Li,Lin Ma

DOI: https://doi.org/10.1038/s41598-023-42689-8

2023-09-21

Abstract:Deep learning's performance on the imbalanced small data is substantially degraded by overfitting. Recurrent neural networks retain better performance in such tasks by constructing dynamical systems for robustness. Synergetic neural network (SNN), a synergetic-based recurrent neural network, has superiorities in eliminating recall errors and pseudo memories, but is subject to frequent association errors. Since the cause remains unclear, most subsequent studies use genetic algorithms to adjust parameters for better accuracy, which occupies the parameter optimization space and hinders task-oriented tuning. To solve the problem and promote SNN's application capability, we propose the modern synergetic neural network (MSNN) model. MSNN solves the association error by correcting the state initialization method in the working process, liberating the parameter optimization space. In addition, MSNN optimizes the attention parameter of the network with the error backpropagation algorithm and the gradient bypass technique to allow the network to be trained jointly with other network layers. The self-learning of the attention parameter empowers the adaptation to the imbalanced sample size, further improving the classification performance. In 75 classification tasks of small UC Irvine Machine Learning Datasets, the average rank of the MSNN achieves the best result compared to 187 neural and non-neural network machine learning methods.
Cost-sensitive hierarchical classification for imbalance classes

Weijie Zheng,Hong Zhao

DOI: https://doi.org/10.1007/s10489-019-01624-z

IF: 5.3

2020-03-04

Applied Intelligence

Abstract:The hierarchical classification with an imbalance class problem is a challenge for in machine learning, and is caused by data with an uneven distribution. Learning from an imbalanced dataset can lead to performance degradation of the classifier. Cost-sensitive learning is a useful solution for handling the gap probability of majority and minority classes. This paper proposes a cost-sensitive hierarchical classification for imbalance classes (CSHCIC), constructing a cost-sensitive factor to balance the relationship between majority and minority classes. First, we divide a large hierarchical classification task into several small subclassification tasks by class hierarchy. Second, we establish a cost-sensitive factor by more precisely using the number of different samples of subclassifications. Then, we calculate the probability of every node using logistic regression. Lastly, we update the cost-sensitive factor using the flexibility factor and the number of samples. The experimental results show that the cost-sensitive hierarchical classification method achieves excellent performance on handling imbalance class datasets. The running time cost of the proposed method is smaller than most state-of-the-art methods.

computer science, artificial intelligence
Towards Balanced Active Learning for Multimodal Classification

Meng Shen,Yizheng Huang,Jianxiong Yin,Heqing Zou,Deepu Rajan,Simon See

DOI: https://doi.org/10.48550/arXiv.2306.08306

2023-06-14

Multimedia

Abstract:Training multimodal networks requires a vast amount of data due to their larger parameter space compared to unimodal networks. Active learning is a widely used technique for reducing data annotation costs by selecting only those samples that could contribute to improving model performance. However, current active learning strategies are mostly designed for unimodal tasks, and when applied to multimodal data, they often result in biased sample selection from the dominant modality. This unfairness hinders balanced multimodal learning, which is crucial for achieving optimal performance. To address this issue, we propose three guidelines for designing a more balanced multimodal active learning strategy. Following these guidelines, a novel approach is proposed to achieve more fair data selection by modulating the gradient embedding with the dominance degree among modalities. Our studies demonstrate that the proposed method achieves more balanced multimodal learning by avoiding greedy sample selection from the dominant modality. Our approach outperforms existing active learning strategies on a variety of multimodal classification tasks. Overall, our work highlights the importance of balancing sample selection in multimodal active learning and provides a practical solution for achieving more balanced active learning for multimodal classification.
A hybrid multi-criteria meta-learner based classifier for imbalanced data

Hasna Chamlal,Hajar Kamel,Tayeb Ouaderhman

DOI: https://doi.org/10.1016/j.knosys.2024.111367

IF: 8.139

2024-01-11

Knowledge-Based Systems

Abstract:Numerous imbalanced datasets exist in modern machine learning dilemmas. Challenges of generalization and fairness stem from the existence of underrepresented classes with sensitive characteristics. Additionally, defining an optimal criterion is crucial when dealing with imbalanced data. To address this, it is common in practice to employ composite metrics and loss functions that evenly distribute the cost across each class. However, relying solely on a single metric or loss function, frequently yields sub-par outcomes due to the tendency of the optimization criteria to be overly responsive to imbalanced data, resulting in excessive adaptation and flawed generalization. In order to overcome these limitations, various heuristics can be applied to optimize the criterion. To this end, we aim at addressing the problem of imbalanced data classification by leveraging a multicriteria ensemble procedure, the Hybrid Multi-criteria Meta-learner (HML). The suggested approach focuses on optimizing precision, recall together with balanced accuracy in the Multi-Objective optimization phase and brings forth optimal ensembles, from which the concluding Meta-learner can be chosen after a clustering step, based on a multicriteria decision-making aspect contingent on the selected metrics. Comprehensive experiments portrayed the effectiveness of HML, offering both quality assurance and addresses a vast array of imbalanced data problems.

computer science, artificial intelligence
Multimodal Fusion Classification Network Based on Distance Confidence Score

ZHENG Dezhong,YANG Yuanyuan,HUANG Haozhe,XIE Zhe,LI Wentao

DOI: https://doi.org/10.16183/j.cnki.jsjtu.2020.186

2022-01-01

Abstract:Multimodal data modeling can effectively overcome the problem of insufficient information in a single mode and can greatly improve the performance of model. However, not much progress has been made in quantifying the confidence of neural network models, especially for multimodal fusion models. This paper proposes a method based on embedding, which calculates the local density estimation in the embedding space by calculating the distance between samples, and then calculates the confidence score of the model. The proposed method is scalable and can be used not only for a single modal model, but also for the confidence measurement of multimodal fusion model. In addition, it can also evaluate and quantify the influences of different modal data on the multimodal fusion model.
A hybrid sampling method for highly imbalanced and overlapped data classification with complex distribution

Yansong Liu,Li Zhu,Lei Ding,He Sui,Wenli Shang

DOI: https://doi.org/10.1016/j.ins.2024.120117

IF: 8.1

2024-01-14

Information Sciences

Abstract:Class imbalance and overlap coupling is the primary cause of performance degradation for classifiers. Unfortunately, it always occurs. In this paper, we propose a hybrid sampling method derived from optimized generative adversarial network and natural neighbor search (HD-GNNS) for this scenario. The new approach considers both the global distribution and local distribution of the minority class to improve the data distribution fundamentally. First, natural neighbor search with Fisher's discriminant ratio is conducted to screen overlapped sample subset and remove noise samples. It effectively overcomes the parameter sensitivity by adaptively determining the search radius. Then, an encoder with squeeze and excite block is introduced into generative adversarial network, and the structure of generative adversarial network is optimized with cross-layer and low-rank matrix. It better captures the distribution characteristics of minority samples in overlapped subset for oversampling. Afterwards, the local density of majority samples in overlapped subset is calculated by the aforementioned natural neighbor search method, and Thornton's Separation Index is used to implement under-sampling adaptively. We evaluate the proposed approach on 1 artificial dataset, 14 UCI datasets and 8 real-word datasets. The experimental results show that the proposed HD-GNNS exhibits more impressive performance compared to other benchmark methods.

computer science, information systems
Multimodal Multi-loss Fusion Network for Sentiment Analysis

Zehui Wu,Ziwei Gong,Jaywon Koo,Julia Hirschberg

2024-06-03

Abstract:This paper investigates the optimal selection and fusion of feature encoders across multiple modalities and combines these in one neural network to improve sentiment detection. We compare different fusion methods and examine the impact of multi-loss training within the multi-modality fusion network, identifying surprisingly important findings relating to subnet performance. We have also found that integrating context significantly enhances model performance. Our best model achieves state-of-the-art performance for three datasets (CMU-MOSI, CMU-MOSEI and CH-SIMS). These results suggest a roadmap toward an optimized feature selection and fusion approach for enhancing sentiment detection in neural networks.

Computation and Language,Artificial Intelligence,Machine Learning,Multimedia
Balanced Multimodal Learning via On-the-fly Gradient Modulation

Xiaokang Peng,Yake Wei,Andong Deng,Dong Wang,Di Hu

DOI: https://doi.org/10.48550/arXiv.2203.15332

2022-03-29

Abstract:Multimodal learning helps to comprehensively understand the world, by integrating different senses. Accordingly, multiple input modalities are expected to boost model performance, but we actually find that they are not fully exploited even when the multimodal model outperforms its uni-modal counterpart. Specifically, in this paper we point out that existing multimodal discriminative models, in which uniform objective is designed for all modalities, could remain under-optimized uni-modal representations, caused by another dominated modality in some scenarios, e.g., sound in blowing wind event, vision in drawing picture event, etc. To alleviate this optimization imbalance, we propose on-the-fly gradient modulation to adaptively control the optimization of each modality, via monitoring the discrepancy of their contribution towards the learning objective. Further, an extra Gaussian noise that changes dynamically is introduced to avoid possible generalization drop caused by gradient modulation. As a result, we achieve considerable improvement over common fusion methods on different multimodal tasks, and this simple strategy can also boost existing multimodal methods, which illustrates its efficacy and versatility. The source code is available at \url{<a class="link-external link-https" href="https://github.com/GeWu-Lab/OGM-GE_CVPR2022" rel="external noopener nofollow">this https URL</a>}.

Computer Vision and Pattern Recognition,Artificial Intelligence
Learning to Rebalance Multi-Modal Optimization by Adaptively Masking Subnetworks

Yang Yang,Hongpeng Pan,Qing-Yuan Jiang,Yi Xu,Jinghui Tang

2024-04-12

Abstract:Multi-modal learning aims to enhance performance by unifying models from various modalities but often faces the "modality imbalance" problem in real data, leading to a bias towards dominant modalities and neglecting others, thereby limiting its overall effectiveness. To address this challenge, the core idea is to balance the optimization of each modality to achieve a joint optimum. Existing approaches often employ a modal-level control mechanism for adjusting the update of each modal parameter. However, such a global-wise updating mechanism ignores the different importance of each parameter. Inspired by subnetwork optimization, we explore a uniform sampling-based optimization strategy and find it more effective than global-wise updating. According to the findings, we further propose a novel importance sampling-based, element-wise joint optimization method, called Adaptively Mask Subnetworks Considering Modal Significance(AMSS). Specifically, we incorporate mutual information rates to determine the modal significance and employ non-uniform adaptive sampling to select foreground subnetworks from each modality for parameter updates, thereby rebalancing multi-modal learning. Additionally, we demonstrate the reliability of the AMSS strategy through convergence analysis. Building upon theoretical insights, we further enhance the multi-modal mask subnetwork strategy using unbiased estimation, referred to as AMSS+. Extensive experiments reveal the superiority of our approach over comparison methods.

Computer Vision and Pattern Recognition,Machine Learning
Multiset Feature Learning for Highly Imbalanced Data Classification

Xiao-Yuan Jing,Xinyu Zhang,Xiaoke Zhu,Fei Wu,Xinge You,Yang Gao,Shiguang Shan,Jing-Yu Yang

DOI: https://doi.org/10.1109/tpami.2019.2929166

IF: 23.6

2021-01-01

IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract:With the expansion of data, increasing imbalanced data has emerged. When the imbalance ratio (IR) of data is high, most existing imbalanced learning methods decline seriously in classification performance. In this paper, we systematically investigate the highly imbalanced data classification problem, and propose an uncorrelated cost-sensitive multiset learning (UCML) approach for it. Specifically, UCML first constructs multiple balanced subsets through random partition, and then employs the multiset feature learning (MFL) to learn discriminant features from the constructed multiset. To enhance the usability of each subset and deal with the non-linearity issue existed in each subset, we further propose a deep metric based UCML (DM-UCML) approach. DM-UCML introduces the generative adversarial network technique into the multiset constructing process, such that each subset can own similar distribution with the original dataset. To cope with the non-linearity issue, DM-UCML integrates deep metric learning with MFL, such that more favorable performance can be achieved. In addition, DM-UCML designs a new discriminant term to enhance the discriminability of learned metrics. Experiments on eight traditional highly class-imbalanced datasets and two large-scale datasets indicate that: the proposed approaches outperform state-of-the-art highly imbalanced learning methods and are more robust to high IR.

computer science, artificial intelligence,engineering, electrical & electronic
An Adaptive Cost-Sensitive Learning and Recursive Denoising Framework for Imbalanced SVM Classification

Lu Jiang,Qi Wang,Yuhang Chang,Jianing Song,Haoyue Fu,Xiaochun Yang

2024-05-16

Abstract:Category imbalance is one of the most popular and important issues in the domain of classification. Emotion classification model trained on imbalanced datasets easily leads to unreliable prediction. The traditional machine learning method tends to favor the majority class, which leads to the lack of minority class information in the model. Moreover, most existing models will produce abnormal sensitivity issues or performance degradation. We propose a robust learning algorithm based on adaptive cost-sensitiveity and recursive denoising, which is a generalized framework and can be incorporated into most stochastic optimization algorithms. The proposed method uses the dynamic kernel distance optimization model between the sample and the decision boundary, which makes full use of the sample's prior information. In addition, we also put forward an effective method to filter noise, the main idea of which is to judge the noise by finding the nearest neighbors of the minority class. In order to evaluate the strength of the proposed method, we not only carry out experiments on standard datasets but also apply it to emotional classification problems with different imbalance rates (IR). Experimental results show that the proposed general framework is superior to traditional methods in accuracy, recall and G-means.

Computer Vision and Pattern Recognition
Multimodal Fusion Convolutional Neural Network Based on sEMG and Accelerometer Signals for Inter-Subject Upper Limb Movement Classification

Anyuan Zhang,Qi Li,Zhenlan Li,Jiming Li

DOI: https://doi.org/10.1109/jsen.2023.3266872

IF: 4.3

2023-01-01

IEEE Sensors Journal

Abstract:The variation in the distributions of recorded data between individuals leads to low classification accuracy. To address this issue, we introduce a multimodal fusion convolutional neural network (MFCNN). This network extracts common information from surface electromyography (sEMG) and accelerometer signals of different subjects using a two-stream convolutional neural network (CNN). To enhance the classification accuracy of a particular subject, a fine-tuning approach was implemented. The performance of the proposed method was assessed in four different scenarios, which include intersubject classification, intersubject classification when training data from multiple subjects, fine-tuned intersubject classification, and fine-tuned intersubject classification when training data from multiple subjects. The results demonstrate that in the intersubject scenario, when multiple subjects are available for training, the MFCNN achieves higher classification accuracy ( ${p} < 0.05$ ) than other neural networks and support vector machines (SVMs) that use sEMG signals [neural network (NN) and SVM], accelerometer signals (accNN and accSVM), sEMG and accelerometer signals [multimodal fusion nerual network (MFNN) and multimodal fusion support vector machine (MFSVM)] as inputs, as well as a CNN that uses sEMG signals as input after fine-tuning. Furthermore, compared with an MFCNN model trained with data from a single subject and an accCNN model trained with data from a single subject or multiple subjects, an MFCNN trained with multiple subjects demonstrated better performance on new subjects after fine-tuning ( ${p} < 0.05$ ). This method can learn common features among different subjects and improve the performance of classification among subjects. Our proposed method demonstrates the innovation of using a multimodal fusion approach and two-stream CNN to improve intersubject classification accuracy in upper limb movements.

engineering, electrical & electronic,instruments & instrumentation,physics, applied
Heterogeneous Hierarchical Fusion Network for Multimodal Sentiment Analysis in Real-World Environments

Ju Huang,Wenkang Chen,Fangyi Wang,Haijun Zhang

DOI: https://doi.org/10.3390/electronics13204137

IF: 2.9

2024-10-22

Electronics

Abstract:Multimodal sentiment analysis models can determine users' sentiments by utilizing rich information from various sources (e.g., textual, visual, and audio). However, there are two key challenges when deploying the model in real-world environments: (1) the limitations of relying on the performance of automatic speech recognition (ASR) models can lead to errors in recognizing sentiment words, which may mislead the sentiment analysis of the textual modality, and (2) variations in information density across modalities complicate the development of a high-quality fusion framework. To address these challenges, this paper proposes a novel Multimodal Sentiment Word Optimization Module and a heterogeneous hierarchical fusion (MSWOHHF) framework. Specifically, the proposed Multimodal Sentiment Word Optimization Module optimizes the sentiment words extracted from the textual modality by the ASR model, thereby reducing sentiment word recognition errors. In the multimodal fusion phase, a heterogeneous hierarchical fusion network architecture is introduced, which first utilizes a Transformer Aggregation Module to fuse the visual and audio modalities, enhancing the high-level semantic features of each modality. A Cross-Attention Fusion Module then integrates the textual modality with the audiovisual fusion. Next, a Feature-Based Attention Fusion Module is proposed that enables fusion by dynamically tuning the weights of both the combined and unimodal representations. It then predicts sentiment polarity using a nonlinear neural network. Finally, the experimental results on the MOSI-SpeechBrain, MOSI-IBM, and MOSI-iFlytek datasets show that the MSWOHHF outperforms several baselines, demonstrating better performance.

engineering, electrical & electronic,computer science, information systems,physics, applied

The grand challenge: facing up to proteomics.

Imbalanced Data Classification Algorithm Based on Integrated Sampling and Ensemble Learning.

Hybrid SVM algorithm oriented to classifying imbalanced datasets

The Balanced Multi-Modal Spiking Neural Networks with Online Loss Adjustment and Time Alignment

CNUSVM: Hybrid CNN-Uneven SVM Model for Imbalanced Visual Learning

CMCI: A Robust Multimodal Fusion Method for Spiking Neural Networks

Self-adaptive cost weights-based support vector machine cost-sensitive ensemble for imbalanced data classification

Adaptive learning cost‐sensitive convolutional neural network

Modern synergetic neural network for imbalanced small data classification

Cost-sensitive hierarchical classification for imbalance classes

Towards Balanced Active Learning for Multimodal Classification

A hybrid multi-criteria meta-learner based classifier for imbalanced data

Multimodal Fusion Classification Network Based on Distance Confidence Score

A hybrid sampling method for highly imbalanced and overlapped data classification with complex distribution

Multimodal Multi-loss Fusion Network for Sentiment Analysis

Balanced Multimodal Learning via On-the-fly Gradient Modulation

Learning to Rebalance Multi-Modal Optimization by Adaptively Masking Subnetworks

Multiset Feature Learning for Highly Imbalanced Data Classification

An Adaptive Cost-Sensitive Learning and Recursive Denoising Framework for Imbalanced SVM Classification

Multimodal Fusion Convolutional Neural Network Based on sEMG and Accelerometer Signals for Inter-Subject Upper Limb Movement Classification

Heterogeneous Hierarchical Fusion Network for Multimodal Sentiment Analysis in Real-World Environments