Code

This page contains information about the whole code stored in the gamma-cat repository.

make.py

There is a command line interface to run the gamma-cat scripts, the make.py file in the top-level folder.

To see the available sub-commands and options:

$ ./make.py --help

To run the full pipeline, i.e. generate all output files and run all checks:

$ ./make.py all

After adding/ changing data in the input folder, one should always execute:

$ ./make.py checks

which checks the format/ structure of the input files.

gammacat package

The make.py command line interface just imports and executes functions and classes from the gammacat Python package (i.e. the .py files in the folder with name gammacat).

We list the modules in gammacat and comment on the code organisation.

The following are more basic modules:

  • utils.py has some helper utility functions (e.g. for JSON / YAML / ECSV I/O)
  • modeling.py has a Parameter and ParameterList class to help process input spatial and spectral source models from the YAML files.
  • info.py has some helpers for versions, filenames, …
  • sed.py has a class to process and validate the spectral energy distributions (SEDs) in the input folder. The SEDs in the output folder can be read directly with gammapy.spectrum.FluxPoints.
  • lightcurve.py has a class to process and validate the lightcurves in the input folder. The lightcurves in the output folder can be read directly with gammapy.time.LightCurve.

In additions there are classes in gammapy.catalog.gammacat that are used in the gammacat scripts to process the data: GammaCatResource, GammaCatResourceIndex, GammaCatDatasetCollection.

Then there is a hierarchy of higher-level modules (that import from the basic modules and modules representing lower-level steps in the processing pipeline):

  • input.py has classes to read / clean up / process the data in the input folder.
  • collection.py has classes to create the files in the output folder (only the dataset files and index files, not the catalog files).
  • cat.py is the code to create the catalog files
  • checks.py is the code to run checks. At the moment the methods there just dispatch to methods called validate or check in lower-level modules (such as gammacat.input), and the actual checks are thus scattered throughout the gammacat modules. There’s also checks on data content in gammacat/tests (which is probably a bad idea, but pytest is convenient to have asserts)

Tests

There is a folder gammacat/tests with some unit tests for the code in the gammacat package, that can be executed via python -m pytest gammacat/tests

We don’t have the relation quite figure out, what goes where:

  • gammacat/tests
  • gammapy/catalog/tests/test_gammacat.py
  • The various check / validate methods throughout gammacat and executed via ./make.py check.

Tools

The following tool is helpful to lint YAML files:

TODO: it’s too picky, showing errors for things that are OK. Figure out how to make it less picky and document that here.

Website build

The gamma-cat website is a static website generated by Python and Sphinx.

We have a Sphinx test page where we can try out things locally and check if they also work on ReadTheDocs. It’s an orphan page, i.e. doesn’t show up for normal users.

We use several Sphinx extensions, and also have our own in gammacat/sphinx/exts.

More info soon … for now this is just a link collection: