Gammapy helps with data management using a multi-layered set of classes. The job of the DataManager and DataStore is to make it easy and fast to locate files and select subsets of observations.
DataManagerrepresents a configuration (usually read from a YAML file) of directories and index files specifying where data is available locally and remotely and in which formats.
DataStorerepresents data files in a given directory and usually consists of two things: a
Tablethat contains the location, content, size, checksum of all files and a
ObservationTablethat contains relevant parameters for each observation (e.g. time, pointing position, …)
- The actual data and IRFs are represented by classes,
The following example demonstrates how data management is done in Gammapy. It uses a test data set, which is available
in the gammapy-extra repository. Please clone this repository and
gammapy-extra/datasets/. The folder
hess-crab4-hd-hap-prod2 contains IRFs and simulated event lists for 4
observations of the Crab nebula. It also contains two index files:
- Observation table
- File table
These files tell gammapy which observations are contained in the data set and where the event list and IRF files are located for each observation (for more information see Data formats).
Exploring the data using the DataStore class works like this
>>> from gammapy.data import DataStore >>> data_store = DataStore.from_dir('$GAMMAPY_EXTRA/datasets/hess-crab4-hd-hap-prod2') >>> data_store.info() Data store summary info: name: noname base_dir: hess-crab4 observations: 4 files: 16 >>> data_store.obs(obs_id=23592).location(hdu_class='events').path(abs_path=False) 'hess-crab4/hess_events_simulated_023592.fits'
In addition, the DataStore class has convenience properties and methods that actually load the data and IRFs and return objects of the appropriate class
>>> event_list = data_store.obs(obs_id=23592).events >>> type(event_list) TODO >>> aeff2d = data_store.obs(obs_id=23592).aeff >>> type(aeff2d) <class 'gammapy.irf.effective_area_table.EffectiveAreaTable2D'> >>> obs.target_radec <SkyCoord (FK5: equinox=J2000.000): (ra, dec) in deg (83.63333333, 22.01444444)>
The data access is even more convenient with a DataManager.It is based one a data registry config file (YAML format)
that specifies where data and index files are located on the user’s machine. In other words, the data registry is
a list of datastores that can be accessed by name. By default, Gammapy looks for data registry config files called
data-register.yaml in the
~/.gammapy folder. Thus, put the following in
in order to proceed with the example.
Now the data access work like this
>>> from gammapy.data import DataManager >>> data_manager = DataManager.from_yaml(DataManager.DEFAULT_CONFIG_FILE) >>> data_manager.store_names ['crab_example'] >>> data_store = data_manager.stores
>>> from gammapy.data import DataStore >>> data_store = DataStore.from_name('crab_example')
Command line tools¶
gammapy-data-manage– Manage data locally and on servers
gammapy-data-browse– A web app to browse local data (stats and quick look plots)