DataStore

class gammapy.data.DataStore(hdu_table=None, obs_table=None, name=None)

Bases: object

IACT data store.

The data selection and access happens using an observation and an HDU index file as described at IACT data storage.

See Data Store or data_iact.html for usage examples.

Parameters:

hdu_table : HDUIndexTable

HDU index table

obs_table : ObservationTable

Observation index table

name : str

Data store name

Examples

Here’s an example how to create a DataStore to access H.E.S.S. data:

>>> from gammapy.data import DataStore
>>> dir = '$GAMMAPY_EXTRA/datasets/hess-crab4-hd-hap-prod2'
>>> data_store = DataStore.from_dir(dir)
>>> data_store.info()

Attributes Summary

DEFAULT_HDU_TABLE	Default HDU table filename.
DEFAULT_NAME	Default data store name.
DEFAULT_OBS_TABLE	Default observation table filename.

Methods Summary

check_available_event_lists([logger])	Check if all event lists are available.
check_integrity([logger])	Check integrity, i.e.
check_observations()	Perform some sanity checks for all observations.
copy_obs(obs_id, outdir[, hdu_class, …])	Create a new DataStore containing a subset of observations.
data_summary([obs_id, summed])	Create a summary Table with HDU size information.
from_all(val)	Try different DataStore constructors.
from_config(config)	Create from a config dict.
from_dir(base_dir[, name])	Create from a directory.
from_files(base_dir[, hdu_table_filename, …])	Construct from HDU and observation index table files.
from_name(name)	Convenience method to look up DataStore from DataManager.
info([file])	Print some info.
load_all([hdu_type, hdu_class])	Load a given file type for all observations.
load_many(obs_ids[, hdu_type, hdu_class])	Load a given file type for certain observations in an observation table.
make_table_of_files([observation_table, …])	Make list of files in the datastore directory.
obs(obs_id)	Access a given DataStoreObservation.
obs_list(obs_id[, skip_missing])	Generate a ObservationList.

Attributes Documentation

DEFAULT_HDU_TABLE = 'hdu-index.fits.gz'

Default HDU table filename.

DEFAULT_NAME = 'noname'

Default data store name.

DEFAULT_OBS_TABLE = 'obs-index.fits.gz'

Default observation table filename.

Methods Documentation

check_available_event_lists(logger=None)

Check if all event lists are available.

TODO: extend this function, or combine e.g. with make_table_of_files.

Returns:

file_available : ndarray

Boolean mask which files are available.

check_integrity(logger=None)

Check integrity, i.e. whether index and observation table match.

check_observations()

Perform some sanity checks for all observations.

Returns:

results : OrderedDict

dictionary containing failure messages for all runs that fail a check.

copy_obs(obs_id, outdir, hdu_class=None, verbose=False, overwrite=False)

Create a new DataStore containing a subset of observations.

Parameters:

obs_id : array-like, ObservationTable

List of observations to copy

outdir : str, Path

Directory for the new store

hdu_class : list of str

see gammapy.data.HDUIndexTable.VALID_HDU_CLASS

verbose : bool

Print copied files

overwrite : bool

Overwrite

data_summary(obs_id=None, summed=False)

Create a summary Table with HDU size information.

Parameters:

obs_id : array-like

Observation IDs to include in the summary

summed : bool

Sum up file size?

classmethod from_all(val)

Try different DataStore constructors.

Currently tried (in this order) - from_dir() - from_name()

Parameters:

val : str

Key to construct DataStore from

classmethod from_config(config)

Create from a config dict.

classmethod from_dir(base_dir, name=None)

Create from a directory.

This assumes that the HDU and observations index tables have the default filename.

classmethod from_files(base_dir, hdu_table_filename=None, obs_table_filename=None, name=None)

Construct from HDU and observation index table files.

classmethod from_name(name)

Convenience method to look up DataStore from DataManager.

info(file=None)

Print some info.

load_all(hdu_type=None, hdu_class=None)

Load a given file type for all observations.

Parameters:

hdu_type : str

HDU type (see VALID_HDU_TYPE)

hdu_class : str

HDU class (see VALID_HDU_CLASS)

Returns:

list : python list of object

Object depends on type, e.g. for events it is a list of EventList.

load_many(obs_ids, hdu_type=None, hdu_class=None)

Load a given file type for certain observations in an observation table.

Parameters:

obs_ids : list

List of observation IDs

hdu_type : str

HDU type (see VALID_HDU_TYPE)

hdu_class : str

HDU class (see VALID_HDU_CLASS)

Returns:

list : list of object

Object depends on type, e.g. for events it is a list of EventList.

make_table_of_files(observation_table=None, filetypes=['events'])

Make list of files in the datastore directory.

Parameters:

observation_table : ObservationTable or None

Observation table (None means select all observations).

filetypes : list of str

File types (TODO: document in a central location and reference from here).

Returns:

table : Table

Table summarising info about files.

obs(obs_id)

Access a given DataStoreObservation.

Parameters:

obs_id : int

Observation ID.

Returns:

obs : DataStoreObservation

Observation container

obs_list(obs_id, skip_missing=False)

Generate a ObservationList.

Parameters:

obs_id : list

Observation IDs.

skip_missing : bool, optional

Skip missing observations, default: False

Returns:

obs : ObservationList

List of DataStoreObservation