- Import scientific data from various sources through one easy Python API.
- Use iterator patterns for each data source (configurable data generators + functions to get next data chunk).
- Skip parser programming and file format handling.
- Enjoy a common namespace for all data and unleash the power of data fusion.
- Handle data distribution in different modes: (1) local download, (2) caching of accessed data, or (3) online stream access
- Easily pull data on cloud servers through Python scripts and facilitate large-scale parallel processing.
- Build on an extensible plattform: Adding access to a new data source only requires addition of its "DataFetcher.py".
- Open source (MIT License)
Supported data sets:
Namespace |
Preview |
Description & Data Source |
|
Astronomy | |||
astro.kepler |
Light curves for stars imaged by the NASA Kepler Space Telescope Source: https://keplerscience.arc.nasa.gov |
||
astro.spectra |
Spectra from the Sloan Digital Sky Survey Source: https://www.sdss.org/dr14/spectro/ |
||
astro.tess.data |
Light curves from TESS Data Alerts Source: https://archive.stsci.edu/prepds/tess-data-alerts/ |
||
astro.tess.simulated |
Simulated light curves from TESS End-to-End 6 Source: https://archive.stsci.edu/prepds/tess-data-alerts/ |
||
astro.voyager |
Data from the Voyager mission. Source: https://spdf.gsfc.nasa.gov/ |
||
Engineering | |||
engineering.la.traffic_counts |
Traffic Count data in Los Angeles. Source: https://data.lacity.org/A-Livable-and-Sustainable-City/LADOT-Traffic-Counts-Summary/94wu-3ps3 |
||
engineering.webcam.mit_sailing |
Images from webcams located at the MIT Sailing Pavilion Source: http://sailing.mit.edu/webcam.php |
||
Finance | |||
finance.timeseries |
Financial time series data retrieved using Alpha Vantage API. Source: https://www.alphavantage.co/ |
||
Geoscience | |||
geo.era_interim |
Era-Interim data at different pressure values from the European Centre for Medium-Range Weather Forecasts accessed through the University Corporation for Atmospheric Research. Source: https://rda.ucar.edu/datasets/ds627.0/ |
||
geo.gldas | Land hydrology model produced by NASA. This version of the data is generated to match the GRACE temporal and spatial characteristics and is available as a complementary data product. Source: https://grace.jpl.nasa.gov/data/get-data/land-water-content |
||
geo.grace |
NASA GRACE Tellus Monthly Mass Grids. 30-day measurements of changes in Earth’s gravity field to quantify equivalent water thickness. Source: https://grace.jpl.nasa.gov/data/get-data/monthly-mass-grids-land |
||
geo.grace.mascon | NASA GRACE Tellus Monthly Mass Grids - Global Mascons. 30-day measurements of changes in Earth’s gravity field to quantify equivalent water thickness. Source: https://grace.jpl.nasa.gov/data/get-data/jpl_global_mascons | ||
geo.groundwater | United States groundwater monitoring wells measuring the depth to water level. Source: https://waterservices.usgs.gov | ||
geo.magnetometer | Data collected at magnetic observatories operated by the U.S. Geological Survey. Source: https://geomag.usgs.gov | ||
geo.mahali.rinex |
Rinex files from the MIT led NSF project studying the Earth’s ionosphere with GPS. Web: http://mahali.mit.edu |
||
geo.mahali.tec |
Total Electron Content from the MIT led NSF project studying the Earth’s ionosphere with GPS. Web:http://mahali.mit.edu |
||
geo.mahali.temperature |
Temperature data from the MIT led NSF project studying the Earth’s ionosphere with GPS. Web: http://mahali.mit.edu |
||
geo.modis |
Spectroradiometer aboard the NASA Terra and Aqua image satellites. Generates approximately daily images of the Earth’s surface. Source:https://modis.gsfc.nasa.gov |
||
geo.pbo |
EarthScope - Plate Boundary Observatory (PBO): Daily GPS displacement time series measurements throughout the United States. Source: http://www.unavco.org/projects/major-projects/pbo/pbo.html |
||
geo.sentinel_1 |
Sentinel-1 TOPSAR data from the European Space Agency retrieved from the Alaska Satellite Facility. Source:https://www.asf.alaska.edu/ |
||
geo.srtm |
Elevation data at a one arc second resolution from the Shuttle Radar Topography Mission (SRTMGL1). Source: https://lpdaac.usgs.gov/dataset_discovery/measures/measures_products_table/srtmgl1_v003 |
||
geo.uavsar |
UAVSAR SLC data from JPL. Source: https://uavsar.jpl.nasa.gov/ |
||
geo.wyoming_sounding |
Sounding data from the University of Wyoming. Source: http://weather.uwyo.edu/upperair/sounding.html |
||
Planetary Science | |||
planetary.ode |
Mars planetary data from PDS Geosciences Node's Orbital Data Explorer. Source: http://pds-geosciences.wustl.edu/default.htm |
||
Solar Science | |||
solar.sdo |
Images from the Solar Dynamics Observatory. Source: https://sdo.gsfc.nasa.gov/ |
pip install scikit-dataaccess
- User Manual: /docs/skdaccess_manual.pdf
- Code documentation (Doxygen): /docs/skdaccess_doxygen.pdf
- Code visualization (treemap): /docs/skdaccess_treemap.png
- Code class diagrams: /docs/class_diagrams
Project lead: Victor Pankratius (MIT)
Contributors: Cody M. Rude, Justin D. Li, David M. Blair, Michael G. Gowanlock, Guillaume Rongier, Victor Pankratius
New contributors welcome! Contact to contribute and add interface code for your own datasets 😄
We acknowledge support from NASA AIST14-NNX15AG84G, NASA AIST16-80NSSC17K0125, NSF ACI-1442997, and NSF AGS-1343967.
Code examples (Jupyter notebooks) for all datasets listed above are available at: /skdaccess/examples