https://github.com/simonsobs/mapcat
Map catalog databases
https://github.com/simonsobs/mapcat
Last synced: about 2 months ago
JSON representation
Map catalog databases
- Host: GitHub
- URL: https://github.com/simonsobs/mapcat
- Owner: simonsobs
- License: mit
- Created: 2025-10-21T18:43:31.000Z (8 months ago)
- Default Branch: main
- Last Pushed: 2026-04-28T17:25:20.000Z (2 months ago)
- Last Synced: 2026-04-28T17:26:39.222Z (2 months ago)
- Language: Python
- Size: 158 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
Simons Observatory Map Catalog
==============================
The Depth-1 and Atomic map catalog for the Simons Observatory.
This repository contains the SQLModel (SQLAlchemy + Pydantic)
types, a client interface, and alembic migration tool for the
database.
Configuration
-------------
`mapcat` is set up to use either PostgreSQL (production) and
SQLite (testing). It can be configured using the `pydantic-settings`
interface defined in `mapcat/helper.py`, with environment variables
using the prefix `MAPCAT_`.
- `export MAPCAT_DATABASE_NAME=mapcat.db`: for SQLite it should be the path to the map
catalog file; If using PostgreSQL, this should be your connection string in the form
`:@:/`. Please do not include
`postgresql://` at the beginning of this variable.
- `export MAPCAT_DATABASE_TYPE=sqlite`: Or `postgresql` if using PostgreSQL.
- `export MAPCAT_DEPTH_ONE_PARENT="."`: Parent directory containing the
Depth-1 maps referred to by the database on this machine.
- `export MAPCAT_ATOMIC_PARENT="."`: Parent directory containing the
Atomic maps referenced by the database on this machine.
- `export MAPCAT_{DEPTH_ONE,ATOMIC}_COADD_PARENT="."`.
Setting up
----------
Mapcat can be installed like any other python package, using your
preferred package manager. We recommend using `uv`:
```
uv pip install mapcat
```
### Creating or Migrating the Database
The database can be migrated (or created, if starting fresh) by using
the:
```
mapcatmigrate
```
command. This runs `alembic upgrade head` with the version of the database
referred to by your currently installed verison of the map catalog.
The database is created in the file or postgres cluster defined in the
configuration above.
Command-line Tools
------------------
We provide a command-line tool for automatically ingesting a directory
of Depth-1 maps stored in the same format as ACT into the database.
Note that the `_info.hdf` files are required, and that these
are not entirely metadata-complete. The command-line script is:
```
actingest --relative-to=/path/to/maps --glob=*/*_map.fits --telescope=act
```
More information on the parameters is available through `actingest -h`.
Registering new Maps
--------------------
New maps can easily be added to the catalog through the SQLAlchemy interface.
It is also wise to include the TOD information used for creating the map,
so that we can later track individual TODs' contributions to various maps.
Note that there does not need to be a one-to-one relationship between
maps and TODs.
```python3
from mapcat.helper import settings
from mapcat.database import DepthOneMapTable, TODDepthOneTable
with settings.session() as session:
tods = [
TODDepthOneTable(
obs_id="obs_...",
pwv=2.3,
...
),
...
]
map = DepthOneMapTable(
map_name="15722/depth1_157221244_lati1_f090",
map_path="15722/depth1_157221244_lati1_f090_map.fits",
ivar_path="15722/depth1_157221244_lati1_f090_ivar.fits",
time_path="15722/depth1_157221244_lati1_f090_time.fits",
tube_slot="i1",
frequency="090",
ctime=157221244.0,
start_time=157220244.0,
stop_time=157222423.0,
tods=tods,
)
session.add(map)
session.commit()
```
By referencing the TOD objects, you automatically create the required
link table items.