https://github.com/robertrosca/nustar

MPhys Project - NuSTAR X-ray Millisecond Variability
https://github.com/robertrosca/nustar

Last synced: 20 days ago
JSON representation

MPhys Project - NuSTAR X-ray Millisecond Variability

• Host: GitHub
• URL: https://github.com/robertrosca/nustar
• Owner: RobertRosca
• Created: 2018-01-19T17:30:39.000Z (almost 7 years ago)
• Default Branch: master
• Last Pushed: 2018-04-26T13:46:47.000Z (over 6 years ago)
• Last Synced: 2024-10-21T22:51:53.733Z (2 months ago)
• Language: Julia
• Size: 282 KB
• Stars: 0
• Watchers: 1
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# NuSTAR

## Notes

Known good observation to use for tests: 80002017002

## Workflow

### Numaster(;local_archive="", local_archive_cl="", local_utility="")

NuSTAR.Numaster() to download the newest data base from the numaster url:

`https://heasarc.gsfc.nasa.gov/FTP/heasarc/dbase/tdat_files/heasarc_numaster.tdat.gz`

Uses directories set in `MiscTools/find_default_path()`

Adds extra columns to the numaster file, all custom columns start with capital

letters, default columns are all lower case. Adds columns for: `Downloaded`,

`Cleaned`, `ValidSci`, `RegSrc`, `RegBkg`

Converts time from Julian to normal readable

Saves CSV file

### Summary(;numaster_path="")

Prints some statistics on what is currently in the `numaster` table

### XMLBatch(;local_archive="default", log_file="", batch_size=100)

Downloading files directly via FTP is a bit slow, can be sped up with simultaneous

downloads, easier to use existing programs to handle that. As such, `XMLBatch()`

takes in the archive paths and a `batch_size` to denote how many new observaions

to queue for download.

It then creates an XML file with can be read into FileZilla's queue to allow for

simultaneous downloading. It does this by calling the `XML(ObsIDs; XML_out_dir="", verbose=false, local_archive="")` function with a queue of multiple `ObsIDs`.

### CalBatch(local_archive="default"; log_file="", batches=4, to_cal=16)

`batches` denotes the number of nupipeline processes to have running simultaneously,

should be equal to or less than the number of CPU cores available.

`to_cal` denotes the **total** number of observations to be calibrated. These are

then split equally (ish) over the number of processes set in `batches`.

Function calls `Calibrate(ObsIDs::Union{Array{String,1}, String})`

### Calibrate(ObsIDs::Union{Array{String,1}, String})

Takes in either a single `ObsID` as a string or an array of strings, calls

the included `run_nupipeline.sh` bash script which then handles the file

transfer to the `/Scratch` folder and the subsequent calibration/file management

### run_nupipeline.sh

Runs nupipeline and sets up temporary directories, scratch folders, notifications

and file movement between scratch and archive directories

Pulls the directory locations from environmental variables set in `.bashrc` or

somewhere else, looks for:'$NU_ARCHIVE', '$NU_ARCHIVE_LIVE', '$NU_ARCHIVE_CL',

and '$NU_ARCHIVE_CL_LIVE'