Our research focuses on providing a fully-imputed map of the worldwide total electron content with high resolution and spatial-temporal smoothness. We fill in the missing values of the original Madrigal TEC maps via estimating the latent feature of each latitude and local time along the 2-D grid and give initial guess of the missing regions based on pre-computed spherical harmonics map. The resulting TEC map has high imputation accuracy and the ease of reproducing.
All data are in HDF5 format and are easy to read using the h5py package in Python. The TEC map is grouped in folders based on years and each file contains a single-day data of 5-min cadence. Each individual TEC map is of size 181*361. and WARNING: 2023-12-01 the data file for 2019-Jan-03 has badly fitted values. Please avoid using it. All other days' files are ready to use.
Sun, H., Hua, Z., Ren, J., Zou, S., Sun, Y., & Chen, Y. (2022). Matrix completion methods for the total electron content video reconstruction. The Annals of Applied Statistics, 16(3), 1333-1358., Sun, H., Chen, Y., Zou, S., Ren, J., Chang, Y., Wang, Z., & Coster, A. (2023). Complete Global Total Electron Content Map Dataset based on a Video Imputation Algorithm VISTA. Scientific Data, in press., and Zou, S., Ren, J., Wang, Z., Sun, H., & Chen, Y. (2021). Impact of storm-enhanced density (SED) on ion upflow fluxes during geomagnetic storm. Frontiers in Astronomy and Space Sciences, 8, 746429.
Our research focuses on providing a fully-imputed map of the worldwide total electron content with high resolution and spatial-temporal smoothness. We fill in the missing values of the original Madrigal TEC maps via estimating the latent feature of each latitude and local time along the 2-D grid and give initial guess of the missing regions based on pre-computed spherical harmonics map. The resulting TEC map has high imputation accuracy and the ease of reproducing. and All data are in HDF5 format and are easy to read using the h5py package in Python. The TEC map is grouped in folders based on years and each file contains a single-day data of 5-min cadence. Each individual TEC map is of size 181*361.
Sun, H., Hua, Z., Ren, J., Zou, S., Sun, Y., & Chen, Y. (2020). Matrix Completion Methods for the Total Electron Content Video Reconstruction. arXiv preprint arXiv:2012.01618. and Zou, S., Ren, J., Wang, Z., Sun, H., & Chen, Y. (2021). Impact of Storm-Enhanced Density (SED) on Ion Upflow Fluxes During Geomagnetic Storm. Frontiers in Astronomy and Space Sciences, 162.
In this work, we perform Global Sensitivity Analysis (GSA) for the background solar wind in order to quantify contributions from uncertainty of different model parameters to the variability of in-situ solar wind speed and density at 1au, both of which have a major impact on CME propagation and strength. Scripts written in the Julia language are used to build the PCE and calculate the sensitivity results. Data is available in csv, NetCDF and JLD files. A `Project.toml` file is included to activate and install all required dependencies (See README for details).
GOES_flare_list: contains a list of more than 10,000 flare events. The list has 6 columns, flare classification, active region number, date, start time end time, emission peak time, GOES_B_flare_list: contains time series data of SDO/HMI SHARP parameters for B class solar flares
, GOES_MX_flare_list: contains time series data of SDO/HMI SHARP parameters for M and X class solar flares, SHARP_B_flare_data_300.hdf5 and SHARP_MX_flare_data_300.hdf5 files contain time series more than 20 physical variables derived from the SDO/HMI SHARP data files. These data are saved at a 12 minute cadence and are used to train the LSTM model., and B_HARPs_CNNencoded_part_xxx.hdf5 and M_X HARPs_CNNencoded_part_xxx.hdf5 include neural network encoded features derived from vector magnetogram images derived from the Solar Dynamics Observatory (SDO) Helioseismic and Magnetic Imager (HMI). These data files typically contains one or two sequences of magnetograms covering an active region for a period of 24h with a 1 hour cadence. We encode each magnetogram with frames of a fixed size of 8x16 with 512 channels.
Chen, Y., Manchester, W., Hero, A., Toth, G., DuFumier, B. Zhou, T., Wang, X., Zhu, H., Sun, Zeyu, Gombosi, T., Identifying Solar Flare Precursors Using Time Series of SDO/HMI Images and SHARP Parameters, Space Weather, 17, 1404–1426. https://doi.org/10.1029/2019SW002214 and Jiao, Z., Chen, Y., Manchester, W. (2020). Data for Solar Flare Intensity Prediction with Machine Learning Models [Data set]. University of Michigan - Deep Blue. https://doi.org/10.7302/b07j-bj08
In this work, we trained gradient boosted trees using XGBoost to predict the SYM-H forecasting using different combinations of solar wind and interplanetary magnetic field (IMF) parameters. Data are in csv and Python pickle formats.
Iong, D., Y. Chen, G. Toth, S. Zou, T. I. Pulkkinen, J. Ren, E. Camporeale, and T. I. Gombosi, New Findings from Explainable SYM-H Forecasting using Gradient Boosting Machines, Space Weather,11, accepted, 2022. https://doi.org/10.1002/essoar.10508063.3
GOES_flare_list: contains a list of more than 12,013 flare events. The list has 6 columns, flare classification, active region number, date, start time end time, emission peak time.
SHARP_data.hdf5 files contain time series of 20 physical variables derived from the SDO/HMI SHARP data files. These data are saved at a 12 minute cadence and are used to train the LSTM model.
Jiao, Z., Sun, H., Wang, X., Manchester, W., Gombosi, T., Hero, A., & Chen, Y. (2020). Solar Flare Intensity Prediction With Machine Learning Models. Space Weather, 18(7), e2020SW002440. https://doi.org/10.1029/2020SW002440 and Chen, Y., & Manchester, W. (2019). Data and Data products for machine learning applied to solar flares [Data set], University of Michigan - Deep Blue. https://doi.org/10.7302/qnsq-cs38