DataStore#

class gammapy.data.DataStore(hdu_table=None, obs_table=None)[source]#

Bases: object

IACT data store.

The data selection and access happens using an observation and an HDU index file as described at Data storage.

Parameters:

hdu_tableHDUIndexTable: HDU index table.
obs_tableObservationTable: Observation index table.

Examples

Here’s an example how to create a DataStore to access H.E.S.S. data:

>>> from gammapy.data import DataStore
>>> data_store = DataStore.from_dir('$GAMMAPY_DATA/hess-dl3-dr1')
>>> data_store.info() 
Data store:
HDU index table:
BASE_DIR: /Users/ASinha/Gammapy-dev/gammapy-data/hess-dl3-dr1
Rows: 630
OBS_ID: 20136 -- 47829
HDU_TYPE: ['aeff', 'bkg', 'edisp', 'events', 'gti', 'psf']
HDU_CLASS: ['aeff_2d', 'bkg_3d', 'edisp_2d', 'events', 'gti', 'psf_table']


Observation table:
Observatory name: 'N/A'
Number of observations: 105

For further usage example see CTA with Gammapy tutorial.

Attributes Summary

`DEFAULT_HDU_TABLE`	Default HDU table filename.
`DEFAULT_OBS_TABLE`	Default observation table filename.
`obs_ids`	Return the sorted obs_ids contained in the datastore.

Methods Summary

`check`([checks])	Check index tables and data files.
`copy_obs`(obs_id, outdir[, hdu_class, ...])	Create a new `DataStore` containing a subset of observations.
`from_dir`(base_dir[, hdu_table_filename, ...])	Create from a directory.
`from_events_files`(events_paths[, irfs_paths])	Create from a list of event filenames.
`from_file`(filename[, hdu_hdu, hdu_obs])	Create a Datastore from a FITS file.
`get_observations`([obs_id, skip_missing, ...])	Generate a `Observations`.
`info`([show])	Print some info.
`obs`(obs_id[, required_irf, require_events])	Access a given `Observation`.

Attributes Documentation

DEFAULT_HDU_TABLE = 'hdu-index.fits.gz'#: Default HDU table filename.

DEFAULT_OBS_TABLE = 'obs-index.fits.gz'#: Default observation table filename.

obs_ids#: Return the sorted obs_ids contained in the datastore.

Methods Documentation

check(checks='all')[source]#

Check index tables and data files.

This is a generator that yields a list of dicts.

copy_obs(obs_id, outdir, hdu_class=None, verbose=False, overwrite=False)[source]#

Create a new DataStore containing a subset of observations.

Parameters:

obs_idarray-like, ObservationTable: List of observations to copy.
outdirstr or Path: Directory for the new store.
hdu_classlist of str, optional: see gammapy.data.HDUIndexTable.VALID_HDU_CLASS.
verbosebool, optional: Print copied files. Default is False.
overwritebool, optional: Overwrite. Default is False.

classmethod from_dir(base_dir, hdu_table_filename=None, obs_table_filename=None)[source]#

Create from a directory.

Parameters:

base_dirstr or Path: Base directory of the data files.
hdu_table_filenamestr or Path, optional: Filename of the HDU index file. May be specified either relative to base_dir or as an absolute path. If None, default is “hdu-index.fits.gz”.
obs_table_filenamestr or Path, optional: Filename of the observation index file. May be specified either relative to base_dir or as an absolute path. If None, default is obs-index.fits.gz.

Returns:

data_storeDataStore: Data store.

Examples

>>> from gammapy.data import DataStore
>>> data_store = DataStore.from_dir('$GAMMAPY_DATA/hess-dl3-dr1')

classmethod from_events_files(events_paths, irfs_paths=None)[source]#

Create from a list of event filenames.

HDU and observation index tables will be created from the EVENTS header.

IRFs are found only if you have a CALDB environment variable set, and if the EVENTS files contain the following keys:

TELESCOP (example: TELESCOP = CTA)
CALDB (example: CALDB = 1dc)
IRF (example: IRF = South_z20_50h)

This method is useful specifically if you want to load data simulated with ctobssim.

Parameters:

events_pathslist of str or Path: List of paths to the events files.
irfs_pathsstr or Path, or list of str or list of Path, optional: Path to the IRFs file. If a list is provided it must be the same length as events_paths. If None the events files have to contain CALDB and IRF header keywords to locate the IRF files, otherwise the IRFs are assumed to be contained in the events files.

Returns:

data_storeDataStore: Data store.

Examples

This is how you can access a single event list:

>>> from gammapy.data import DataStore
>>> import os
>>> os.environ["CALDB"] = os.environ["GAMMAPY_DATA"] + "/cta-1dc/caldb"
>>> path = "$GAMMAPY_DATA/cta-1dc/data/baseline/gps/gps_baseline_110380.fits"
>>> data_store = DataStore.from_events_files([path])
>>> observations = data_store.get_observations()

You can now analyse this data as usual (see any Gammapy tutorial).

If you have multiple event files, you have to make the list. Here’s an example using Path.glob to get a list of all events files in a given folder:

>>> import os
>>> from pathlib import Path
>>> path = Path(os.environ["GAMMAPY_DATA"]) / "cta-1dc/data"
>>> paths = list(path.rglob("*.fits"))
>>> data_store = DataStore.from_events_files(paths)
>>> observations = data_store.get_observations()

>>> #Note that you have a lot of flexibility to select the observations you want,
>>> # by having a few lines of custom code to prepare ``paths``, or to select a
>>> # subset via a method on the ``data_store`` or the ``observations`` objects.
>>> # If you want to generate HDU and observation index files, write the tables to disk::

>>> data_store.hdu_table.write("hdu-index.fits.gz") 
>>> data_store.obs_table.write("obs-index.fits.gz") 

classmethod from_file(filename, hdu_hdu='HDU_INDEX', hdu_obs='OBS_INDEX')[source]#

Create a Datastore from a FITS file.

The FITS file must contain both index files.

Parameters:

filenamestr or Path: FITS filename.
hdu_hdustr or int, optional: FITS HDU name or number for the HDU index table. Default is “HDU_INDEX”.
hdu_obsstr or int, optional: FITS HDU name or number for the observation index table. Default is “OBS_INDEX”.

Returns:

data_storeDataStore: Data store.

get_observations(obs_id=None, skip_missing=False, required_irf='full-enclosure', require_events=True)[source]#

Generate a Observations.

Parameters:

obs_idlist, optional

Observation IDs. If None, default is all observations ordered by OBS_ID are returned. This is not necessarily the order in the obs_table.

skip_missingbool, optional

Skip missing observations. Default is False.

required_irflist of str or str, optional

Runs will be added to the list of observations only if the required HDUs are present. Otherwise, the given run will be skipped The list can include the following options:

"events" : Events
"gti" : Good time intervals
"aeff" : Effective area
"bkg" : Background
"edisp" : Energy dispersion
"psf" : Point Spread Function
"rad_max" : Maximal radius

Alternatively single string can be used as shortcut:

"full-enclosure" : includes ["events", "gti", "aeff", "edisp", "psf", "bkg"]
"point-like" : includes ["events", "gti", "aeff", "edisp"]
"all-optional" : no HDUs are required, only warnings will be emitted for missing HDUs among all possibilities.

Default is "full-enclosure".

require_eventsbool, optional

Require events and gti table or not. Default is True.

Returns:

observationsObservations: Container holding a list of Observation.

Notes

The progress bar can be displayed for this function.

info(show=True)[source]#: Print some info.

obs(obs_id, required_irf='full-enclosure', require_events=True)[source]#

Access a given Observation.

Parameters:

obs_idint

Observation ID.

required_irflist of str or str, optional

The list can include the following options:

"events" : Events
"gti" : Good time intervals
"aeff" : Effective area
"bkg" : Background
"edisp" : Energy dispersion
"psf" : Point Spread Function
"rad_max" : Maximal radius

Alternatively single string can be used as shortcut:

"full-enclosure" : includes ["events", "gti", "aeff", "edisp", "psf", "bkg"]
"point-like" : includes ["events", "gti", "aeff", "edisp"]

Default is "full-enclosure".

require_eventsbool, optional

Require events and gti table or not. Default is True.

Returns:

observationObservation: Observation container.