Work Description

Date: 5 December, 2024

Dataset Title: Decent Estimate of CME Arrival time from a Data-assimilated Ensemble in the Alfvén Wave Solar atmosphere Model

Dataset Creators: Chen, H., Sachdeva, N., Huang, Z., van der Holst, B., Manchester, W., Jivani, A., Zou, S., Chen, Y., Huan, X., Toth, G.

Dataset Contact: Hongfan Chen chenhf@umich.edu

Funding: This work is supported by the National Science Foundation (NSF) under grant number PHY-2027555: SWQU: NextGen Space Weather Modeling Framework Using Data, Physics and Uncertainty Quantification. W. Manchester was partially supported by NASA grant 80NSSC21K1685, and is also funded by NSF Solar Terrestrial grant No. 2323303 and NASA LWS grants 80NSSC22K0892 and 80NSSC24K1104.

Key Points:
- Coronal white light observations are used to constrain an ensemble of coronal mass ejection simulations.
- Automated methods are developed to identify and compare important features in observed and simulated white light images.
- We find a marked improvement of CME arrival time predictions with the new approach for multiple events.

Research Overview:
Accurately predicting the arrival time of coronal mass ejections (CMEs) continues to be a challenge in space weather forecasting. To enhance the accuracy and reliability of predictions, data assimilation techniques can be employed. In this study, we investigate assimilating CME simulations with SOHO LASCO white-light solar corona observations. By establishing a correlation between CME speed extracted from remote white-light images and CME arrival time at Earth, we are able to perform data assimilation at an early stage of the CME simulation process. This enables us to effectively constrain the simulation and improve the overall quality of the prediction.

Methodology:
The data are processed model output from ensemble CME simulations conducted with the Space Weather Modeling Framework (SWMF). The raw simulation data are stored in SolsticeDisk and are available upon request.

Files contained here:
The folders show divisions based on contents. The data folder contains processed simulation output and analysis output. The figure folder contains the raw figures. The figure_paper folder contains the final figures used in the paper. The scripts output contains scripts used to implement the correlation analysis and data assimilation procedure introduced in the paper. The detailed information is described below:
The data folder: this folder contains all data used in our analysis.
- 2014_09_10_CME1: data related to CME1
- 2015_03_15_CME2: data related to CME2
- 2017_07_14_CME3: data related to CME3
- Sensitivity: this folder contains the results of sensitivity analysis.
** CR2192_BART_params_120runs: the optimal hyperparameters of the fitted Bayesian additive regression tree model.
** VarImpo_CR2192_120runs.csv: the variable importance score of EEGGL parameters calculated from the first batch of CME simulations.
- OBSERVATIONS: this folder contains LASCO SOHO observations as well as CDAW catalog data.
- UQ_X: this folder contains intermediate output of the data assimilation procedure, including posterior samples from Metropolis-Hastings algorithm and the overshoot-arrival time data.
- T_Event_CV_Bayesian0.05_correction.csv: the prediction error of CME arrival time before and after the data assimilation procedure.
The figure folder: this folder contains all figures used in the paper.
The scripts folder: this folder contains the scripts used to implement the method. They are ordered in 5 steps.
- Step1_background_selection: this folder contains data and scripts for the background selection based on the curve distance calculated from a 10-day time window.
- Step2_ArrivalEstimation_BgCorctn: the scripts are in R and python.
** CoM_Demo.R: change point detection algorithm.
** QoIs_Ushift_120runs_withCorrection.R: CME arrival time estimation with background correction for the first batch of simulations.
** QoIs_Ushift_24by3runs_withCorrection.R: CME arrival time estimation with background correction for the second batch of simulations.
** Bg_get_vobs_vsim: this folders contains data for the background solar wind speed.
- Step3_SensitivityAnalysis
** BART_ARR_2192_120runs.R: Sensitivity analysis using the first batch of CME simulations of CME3.
** Design.R: MaxPro Design visualization of 120 runs of CME3.
** discard.R: discarded runs of the first batch of CME3.
** iH_B_scatter.R: Scatter plots, Helicity versus BStrength.
** iHelicity_comparison.R: Comparison between quantities of interest with flipped helicity.
- Step4_WLSpeedEstimation
** CR2192_edge_morph_120runs.py: tracking CME edge for the first batch of simulations. This script requires raw simulation data, which is not included in this repository due to the storage size. It can be made available upon request.
** CR****_edge_morph_24runs.py: tracking CME edge for the second batch of simulations. This script requires raw simulation data, which is not included in this repository due to the storage size. It can be made available upon request.
** CR2161_SIM_OBS_DEMO.py: the demo for edge detection algorithm.
** Kmeans_CR****_obs_CDAW.py: the comparison between our estimate and that of the CDAW catalog.
** vbar_linearfit_2192_120runs.py: calculating velocity from CME edges using the first batch of simulations.
** vbar_linearfit_202403.py: calculating velocity from CME edges using the second batch of simulations.
- Step5_UQ_DA
** UQ_step1_etaY.R: the first step of UQ. Cleaning data to create overshoot-arrival time pairs.
** UQ_step2_MH.R: the second step of UQ. Metropolis-Hastings sampling and posterior samples.
** DA_correlation_probpred.R: correlation analysis and data assimilation procedure.
- OtherFigures
** Image_Merge.py: merge figures
** Stretched_torus.py: schematic of a flux rope.

Software Used:
- R packages: tidyverse, ncdf4, Hmisc, scales, ggpubr, gridExtra, bartMachine, latex2exp, units, reshape2
- Python packages: sunpy, matplotlib, numpy, pandas, pylab, math, scipy, os, time, re, cv2, sklearn, PIL, sys
- Julia packages: NetCDF, Plots, ColorSchemes, Dates, JLD, CSV, DataFrames, Statistics, LaTeXStrings, PyCall, Colors

Note: There is no specific requirement on the version of packages. The latest versions of these packages should work properly. Prior to running the scripts, please ensure that the path variable in each script is set to your local path of the 'PaperWL_Materials' folder if the script is written in R or Python.

Related publication(s):
Chen, H., et al. (2024). Decent Estimate of CME Arrival time from a Data-assimilated Ensemble in the Alfvén Wave Solar atmosphere Model (DECADE-AWSoM). Forthcoming

Use and Access:
This data set is made available under a Creative Commons Public Domain license (CC0 1.0).

To Cite Data:
Chen, H., Sachdeva, N., Huang, Z., van der Holst, B., Manchester, W., Jivani, A., Zou, S., Chen, Y., Huan, X., Toth, G. Decent Estimate of CME Arrival time from a Data-assimilated Ensemble in the Alfvén Wave Solar atmosphere Model [Data set], University of Michigan - Deep Blue Data.