Abstract:Most of the classification methods of urban functional areas nowadays are only based on single source data analysis and modeling, which can not make full use of the multi-scale and multi-source data that is easy to obtain. Therefore, this paper proposed a classification model of urban functional areas based on multi-modal machine learning, by analyzing regional remote sensing images and behavior data of visitors in the area, using the combination of supervised methods extracted the deep-seated features and relationships of kinds of data, filtered and merged the overall and local features of the data. The model used dual branch neural network combining SE-ResNeXt and Dual Path Network (DPN) to automatically mined and fused the overall characteristics of multi-source data, and used the designed feature engineering to deeply mine the behavior data of users to obtain more association information, then combined the algorithm based on Gradient Boosting Decision Tree to learn the characteristics of different levels and obtained the classification probability for different levels of features. Finally, we continued to use the algorithm based on the Gradient Boosting Decision Tree to learn the probability distribution of different levels of features to obtain the final prediction results of urban functional area classification. Through the analysis and experimental verification of real data sets, the results showed that MM-UrbanFAC model can effectively integrate the features of multi-modal data. Compared with a single classifier, the integration framework based on gradient lifting tree improved the prediction performance, this method can effectively integrate the results of multiple models and accurately classify urban functional areas, and the model can provide reference for tourism recommendation, urban land planning and urban construction.

Remote Sensing and Time Series Data Fused Multimodal Prediction Model Based on Interaction Analysis

Multimodal-Temporal Fusion: Blending Multimodal Remote Sensing Images to Generate Image Series With High Temporal Resolution

A multimodal fusion framework for urban scene understanding and functional identification using geospatial data

MM-UrbanFAC: Urban Functional Area Classification Model Based on Multimodal Machine Learning

A Cyclic Information–Interaction Model for Remote Sensing Image Segmentation

Deep Multimodal Data Fusion

Cross-Modal Sentiment Sensing with Visual-Augmented Representation and Diverse Decision Fusion

Consumer Intention Recognition and Behavior Prediction of Social E-commerce Users Based on Multimodal Fusion

Multimodal Language Analysis with Recurrent Multistage Fusion

Towards Effective Fusion and Forecasting of Multimodal Spatio-temporal Data for Smart Mobility

Dual-View Multimodal Interaction in Multimodal Sentiment Analysis

A cross modal hierarchical fusion multimodal sentiment analysis method based on multi-task learning

A Deep-Learning-Based Multimodal Data Fusion Framework for Urban Region Function Recognition

Multimodal Sentiment Analysis Using Multi-tensor Fusion Network with Cross-modal Modeling

Multimodal Remote Sensing Data Classification Based on Gaussian Mixture Variational Dynamic Fusion Network

Modality Fusion Vision Transformer for Hyperspectral and LiDAR Data Collaborative Classification

Similarity Guided Multimodal Fusion Transformer for Semantic Location Prediction in Social Media

Two-Stage Spatial Mapping for Multimodal Data Fusion in Mobile Crowd Sensing

Multi-modal Deep Analysis for Multimedia

Feature Extraction Network with Attention Mechanism for Data Enhancement and Recombination Fusion for Multimodal Sentiment Analysis

A Multimodal Data Fusion Model for Accurate and Interpretable Urban Land Use Mapping with Uncertainty Analysis