Abstract:All artificial intelligence models today require preprocessed and cleaned data to work properly. This crucial step depends on the quality of the data analysis being done. The Space Weather community increased its use of AI in the past few years, but a thorough data analysis addressing all the potential issues is not always performed beforehand. Here is an analysis of a largely used dataset: Level-2 Advanced Composition Explorer’s SWEPAM and MAG measurements from 1998 to 2021 by the ACE Science Center. This work contains guidelines and highlights issues in the ACE data that are likely to be found in other space weather datasets: missing values, inconsistency in distributions, hidden information in statistics, etc. Amongst all specificities of this data, the following can seriously impact the use of algorithms: Histograms are not uniform distributions at all, but sometime Gaussian or Laplacian. Algorithms will be inconsistent in the learning samples as some rare cases will be underrepresented. Gaussian distributions could be overly brought by Gaussian noise from measurements and the signal-to-noise ratio is difficult to estimate. Models will not be reproducible from year to year due to high changes in histograms over time. This high dependence on the solar cycle suggests that one should have at least 11 consecutive years of data to train the algorithm. Rounding of ion temperatures values to different orders of magnitude throughout the data, (probably due to a fixed number of bits on which measurements are coded) will bias the model by wrongly over-representing or under-representing some values. There is an extensive number of missing values (e.g., 41.59% for ion density) that cannot be implemented without pre-processing. Each possible pre-processing is different and subjective depending on one’s underlying objectives A linear model will not be able to accurately model the data. Our linear analysis (e.g., PCA), struggles to explain the data and their relationships. However, non-linear relationships between data seem to exist. Data seem cyclic: we witness the apparition of the solar cycle and the synodic rotation period of the Sun when looking at autocorrelations. Some suggestions are given to address the issues described to enable usage of the dataset despite these challenges.

Scope and limitations of ad hoc neural network reconstructions of solar wind parameters

On general correlation between 3D solar wind speed and density model and solar proxies

Charge States and FIP Bias of the Solar Wind from Coronal Holes, Active Regions, and Quiet Sun

Comparative Analysis of the Solar Wind: Modeling Charge State Distributions in the Heliosphere

Assessing the Performance of the ADAPT and AFT Flux Transport Models Using In Situ Measurements from Multiple Satellites

Assessing the Performance of the ADAPT and AFT Flux Transport Models Using In-Situ Measurements From Multiple Satellites

Towards an AI-based understanding of the solar wind: A critical data analysis of ACE data

Solar wind data analysis aided by synthetic modeling: A better understanding of plasma frame variations from temporal data

Solar wind data analysis aided by synthetic modeling: a better understanding of plasma-frame variations from temporal data

A comparison study of extrapolation models and empirical relations in forecasting solar wind

The solar wind from a stellar perspective: how do low-resolution data impact the determination of wind properties?

Objectively Determining States of the Solar Wind Using Machine Learning

Matching temporal signatures of solar features to their corresponding solar wind outflows

Solar Wind Anomalies at 1 au and Their Associations with Large-scale Structures

Forecasting High-Speed Solar Wind Streams from Solar Images

Ensemble Forecasts of Solar Wind Connectivity to 1 Rs using ADAPT-WSA

NIRwave: A wave-turbulence-driven solar wind model constrained by PSP observations

Constraining solar wind transport model parameters using Bayesian analysis

SynCOM: An Empirical Model for High-Resolution Simulations of Transient Solar Wind Flows

Improvement of Plasma Sheet Neural Network Accuracy With Inclusion of Physical Information

Uncertainty Estimates of Solar Wind Prediction Using HMI Photospheric Vector and Spatial Standard Deviation Synoptic Maps