Our research focuses on providing a fully-imputed map of the worldwide total electron content with high resolution and spatial-temporal smoothness. We fill in the missing values of the original Madrigal TEC maps via estimating the latent feature of each latitude and local time along the 2-D grid and give initial guess of the missing regions based on pre-computed spherical harmonics map. The resulting TEC map has high imputation accuracy and the ease of reproducing.
All data are in HDF5 format and are easy to read using the h5py package in Python. The TEC map is grouped in folders based on years and each file contains a single-day data of 5-min cadence. Each individual TEC map is of size 181*361. and WARNING: 2023-12-01 the data file for 2019-Jan-03 has badly fitted values. Please avoid using it. All other days' files are ready to use.
Sun, H., Hua, Z., Ren, J., Zou, S., Sun, Y., & Chen, Y. (2022). Matrix completion methods for the total electron content video reconstruction. The Annals of Applied Statistics, 16(3), 1333-1358., Sun, H., Chen, Y., Zou, S., Ren, J., Chang, Y., Wang, Z., & Coster, A. (2023). Complete Global Total Electron Content Map Dataset based on a Video Imputation Algorithm VISTA. Scientific Data, in press., and Zou, S., Ren, J., Wang, Z., Sun, H., & Chen, Y. (2021). Impact of storm-enhanced density (SED) on ion upflow fluxes during geomagnetic storm. Frontiers in Astronomy and Space Sciences, 8, 746429.
Our research focuses on providing a fully-imputed map of the worldwide total electron content with high resolution and spatial-temporal smoothness. We fill in the missing values of the original Madrigal TEC maps via estimating the latent feature of each latitude and local time along the 2-D grid and give initial guess of the missing regions based on pre-computed spherical harmonics map. The resulting TEC map has high imputation accuracy and the ease of reproducing. and All data are in HDF5 format and are easy to read using the h5py package in Python. The TEC map is grouped in folders based on years and each file contains a single-day data of 5-min cadence. Each individual TEC map is of size 181*361.
Sun, H., Hua, Z., Ren, J., Zou, S., Sun, Y., & Chen, Y. (2020). Matrix Completion Methods for the Total Electron Content Video Reconstruction. arXiv preprint arXiv:2012.01618. and Zou, S., Ren, J., Wang, Z., Sun, H., & Chen, Y. (2021). Impact of Storm-Enhanced Density (SED) on Ion Upflow Fluxes During Geomagnetic Storm. Frontiers in Astronomy and Space Sciences, 162.
In this work, we trained gradient boosted trees using XGBoost to predict the SYM-H forecasting using different combinations of solar wind and interplanetary magnetic field (IMF) parameters. Data are in csv and Python pickle formats.
Iong, D., Y. Chen, G. Toth, S. Zou, T. I. Pulkkinen, J. Ren, E. Camporeale, and T. I. Gombosi, New Findings from Explainable SYM-H Forecasting using Gradient Boosting Machines, Space Weather,11, accepted, 2022. https://doi.org/10.1002/essoar.10508063.3