DataStore¶
-
class
gammapy.data.
DataStore
(hdu_table=None, obs_table=None, name=None)[source]¶ Bases:
object
IACT data store.
The data selection and access happens using an observation and an HDU index file as described at IACT data storage.
See Data Store or data_iact.html for usage examples.
Parameters: hdu_table :
HDUIndexTable
HDU index table
obs_table :
ObservationTable
Observation index table
name : str
Data store name
Examples
Here’s an example how to create a
DataStore
to access H.E.S.S. data:>>> from gammapy.data import DataStore >>> dir = '$GAMMAPY_EXTRA/datasets/hess-crab4-hd-hap-prod2' >>> data_store = DataStore.from_dir(dir) >>> data_store.info()
Attributes Summary
DEFAULT_HDU_TABLE
Default HDU table filename. DEFAULT_NAME
Default data store name. DEFAULT_OBS_TABLE
Default observation table filename. Methods Summary
check_available_event_lists
([logger])Check if all event lists are available. check_integrity
([logger])Check integrity, i.e. check_observations
()Perform some sanity checks for all observations. copy_obs
(obs_id, outdir[, hdu_class, …])Create a new DataStore
containing a subset of observations.data_summary
([obs_id, summed])Create a summary Table
with HDU size information.from_all
(val)Try different DataStore constructors. from_config
(config)Create from a config dict. from_dir
(base_dir[, name])Create from a directory. from_files
(base_dir[, hdu_table_filename, …])Construct from HDU and observation index table files. from_name
(name)Convenience method to look up DataStore from DataManager. info
([file])Print some info. load_all
([hdu_type, hdu_class])Load a given file type for all observations. load_many
(obs_ids[, hdu_type, hdu_class])Load a given file type for certain observations in an observation table. make_table_of_files
([observation_table, …])Make list of files in the datastore directory. obs
(obs_id)Access a given DataStoreObservation
.obs_list
(obs_id[, skip_missing])Generate a ObservationList
.Attributes Documentation
-
DEFAULT_HDU_TABLE
= 'hdu-index.fits.gz'¶ Default HDU table filename.
-
DEFAULT_NAME
= 'noname'¶ Default data store name.
-
DEFAULT_OBS_TABLE
= 'obs-index.fits.gz'¶ Default observation table filename.
Methods Documentation
-
check_available_event_lists
(logger=None)[source]¶ Check if all event lists are available.
TODO: extend this function, or combine e.g. with
make_table_of_files
.Returns: file_available :
ndarray
Boolean mask which files are available.
-
check_integrity
(logger=None)[source]¶ Check integrity, i.e. whether index and observation table match.
-
check_observations
()[source]¶ Perform some sanity checks for all observations.
Returns: results : OrderedDict
dictionary containing failure messages for all runs that fail a check.
-
copy_obs
(obs_id, outdir, hdu_class=None, verbose=False, overwrite=False)[source]¶ Create a new
DataStore
containing a subset of observations.Parameters: obs_id : array-like,
ObservationTable
List of observations to copy
outdir : str, Path
Directory for the new store
hdu_class : list of str
verbose : bool
Print copied files
overwrite : bool
Overwrite
-
data_summary
(obs_id=None, summed=False)[source]¶ Create a summary
Table
with HDU size information.Parameters: obs_id : array-like
Observation IDs to include in the summary
summed : bool
Sum up file size?
-
classmethod
from_all
(val)[source]¶ Try different DataStore constructors.
Currently tried (in this order) -
from_dir()
-from_name()
Parameters: val : str
Key to construct DataStore from
-
classmethod
from_dir
(base_dir, name=None)[source]¶ Create from a directory.
This assumes that the HDU and observations index tables have the default filename.
-
classmethod
from_files
(base_dir, hdu_table_filename=None, obs_table_filename=None, name=None)[source]¶ Construct from HDU and observation index table files.
-
load_all
(hdu_type=None, hdu_class=None)[source]¶ Load a given file type for all observations.
Parameters: hdu_type : str
HDU type (see
VALID_HDU_TYPE
)hdu_class : str
HDU class (see
VALID_HDU_CLASS
)Returns: list : python list of object
Object depends on type, e.g. for
events
it is a list ofEventList
.
-
load_many
(obs_ids, hdu_type=None, hdu_class=None)[source]¶ Load a given file type for certain observations in an observation table.
Parameters: obs_ids : list
List of observation IDs
hdu_type : str
HDU type (see
VALID_HDU_TYPE
)hdu_class : str
HDU class (see
VALID_HDU_CLASS
)Returns: list : list of object
Object depends on type, e.g. for
events
it is a list ofEventList
.
-
make_table_of_files
(observation_table=None, filetypes=['events'])[source]¶ Make list of files in the datastore directory.
Parameters: observation_table :
ObservationTable
or NoneObservation table (
None
means select all observations).filetypes : list of str
File types (TODO: document in a central location and reference from here).
Returns: table :
Table
Table summarising info about files.
-
obs
(obs_id)[source]¶ Access a given
DataStoreObservation
.Parameters: obs_id : int
Observation ID.
Returns: obs :
DataStoreObservation
Observation container
-
obs_list
(obs_id, skip_missing=False)[source]¶ Generate a
ObservationList
.Parameters: obs_id : list
Observation IDs.
skip_missing : bool, optional
Skip missing observations, default: False
Returns: obs :
ObservationList
List of
DataStoreObservation
-