Abstract:Satellite Image Situation Awareness (SISA) is a task that generates semantic situations from satellite images automatically. It requires not only the position and basic attributions (color, size, etc.) of targets but also the relationships (counting, relative position, existence, comparison, etc.) among them and realization of the situation analysis rules. We propose a novel framework which consists of the Background Process, Visual Question Answering (VQA), and Association Rules Set (ARS), in which, the Background Process deals with the situational map, the VQA and ARS identifies the relationships through answering a set of questions on SISA. To verify the performance of our method, we build the evaluation dataset based on CLEVR. Experiments demonstrate that our approach outperforms the traditional SISA systems on accuracy and automaticity. To the best of our knowledge, we are the first to solve SA problem using VQA method. The meaning of our research are: (1) We provide the possibility that SISA can be accomplished through VQA (without precise scene graph). (2) We broaden the application of VQA.

Semantic Situation Extraction from Satellite Image Based on Neural Networks.