{"id":23804260,"url":"https://github.com/attipaci/crush","last_synced_at":"2026-04-26T04:30:17.231Z","repository":{"id":128809356,"uuid":"110775237","full_name":"attipaci/crush","owner":"attipaci","description":"Data reduction and imaging for select astronomical cameras","archived":false,"fork":false,"pushed_at":"2024-09-16T07:47:56.000Z","size":12114,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-01T22:39:53.581Z","etag":null,"topics":["astronomy","bolometers","data-reduction","imaging","infrared","java","polarimetry","submillimeter"],"latest_commit_sha":null,"homepage":"http://www.sigmyne.com/crush","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/attipaci.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"attipaci","patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"lfx_crowdfunding":null,"polar":null,"buy_me_a_coffee":null,"thanks_dev":null,"custom":null}},"created_at":"2017-11-15T02:53:46.000Z","updated_at":"2024-09-16T07:48:01.000Z","dependencies_parsed_at":"2023-07-09T11:38:43.316Z","dependency_job_id":null,"html_url":"https://github.com/attipaci/crush","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/attipaci%2Fcrush","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/attipaci%2Fcrush/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/attipaci%2Fcrush/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/attipaci%2Fcrush/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/attipaci","download_url":"https://codeload.github.com/attipaci/crush/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240054101,"owners_count":19740766,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["astronomy","bolometers","data-reduction","imaging","infrared","java","polarimetry","submillimeter"],"created_at":"2025-01-01T22:39:58.534Z","updated_at":"2026-04-26T04:30:17.197Z","avatar_url":"https://github.com/attipaci.png","language":"Java","funding_links":["https://github.com/sponsors/attipaci"],"categories":[],"sub_categories":[],"readme":"# CRUSH\n\nAuthor: Attila Kovacs \u003cattila[AT]sigmyne.com\u003e  \nLast updated: 12 July 2019\n\n\n----------------------------------------------------------------------------\n\n\n#### Table of Contents\n\n1. __Getting Started__\n    - 1.1. Installation\n    - 1.2. (optional) Java Runtime Configuration\n    - 1.3. CRUSH Pipeline Configuration\n    - 1.4. The Tools of the CRUSH Suite\n    - 1.5 Quick Start\n    - 1.6 A Brief Description of What CRUSH Does and How...\n    - 1.7 Command-line Options and Scripting Support\n    - 1.8 Making Sense of the Console Output\n    - 1.9 Examples\n\n2. __Advanced Topics__\n    - 2.1 Pointing Corrections\n    - 2.2 Pixelization and Smoothing\n    - 2.3 Recovery of Extended Emission\n    - 2.4 Filtering corrections, transfer functions, etc.\n    - 2.5 Image Processing Post-Reduction\n    - 2.6 Reducing Very Large Datasets (to do...)\n\n3. __Advanced Configuration__\n    - 3.1 Commands\n    - 3.2 Pipeline Configuration\n    - 3.3 Source (Map) Configuration\n    - 3.4 Scan Configuration\n    - 3.5 Instrument Configuration\n    - 3.6 Advanced startup environment and Java configuration\n\n4. __Correlated Signals__ \n    - 4.1 Modes and Modalities\n    - 4.2 Removing Correlated Signals\n    - 4.3 User-defined Modalities\n\n5. __Custom Logging Support__\n    - 5.1 Log Files and Versioning\n    - 5.2 Decimal Formatting of Values\n    - 5.3 Loggable Quantities\n\n6. __Supported Cameras__\n\n7. __Future Developments (A Wish List...)__\n    - 7.1 Support for Heterodyne Receivers and Interferometry\n    - 7.2 Interactive Reductions\n    - 7.3 Distributed and GPU Computing\n    - 7.4 Graphical User-Interface (GUI)\n\n8. __Further information__\n\n\n----------------------------------------------------------------------------\n\n\n\n\n## 1. Getting Started\n\nCRUSH-2 is a reduction and imaging package for various astronomical imaging\narrays. Version 2 provides a unified platform for reducing data from\nvirtually any fast-sampling scanning camera. \n\n\n\n### 1.1. Installation\n\n#### Install Java (if not already installed)\n   \n  Download Java, e.g. from www.java.com. If you have Java already, check that \n  it is version 1.8.0 (a.k.a. Java 8) or later, by typing:\n\n    \u003e java -version\n\n  Note, that The GNU java a.k.a. gij (default on some older RedHat and Fedora\n  systems) is painfully slow and unreliable, and will not always run CRUSH\n  correctly. If you need Java, you can download the latest JRE from\n  \n   https://java.com\n\n\n#### Download CRUSH\n\n  You can get the latest version of CRUSH from:\n\n   http://www.sigmyne.com/crush \n\n  Linux users can grab one of the distribution packages to install CRUSH via\n  a package manager. Packages are provided for both RPM-based distributions \n  (like Fedora, Redhat, CentOS, or SUSE), and Debian-based distributions (e.g.\n  Ubuntu, Mint). Both will install CRUSH in `/usr/share/crush`. Note, if you \n  use one of these packages you will need root privileges (e.g. via `sudo`) on\n  the machine. Unprivileged users should install using the tarball.\n\n  For all others, CRUSH is distributed as a gzipped tarball (or as a zip \n  archive). Simply unpack it in the directory, where you want the crush to \n  reside. \n\n  \n#### A. Installation from tarball (POSIX/UNIX, incl. Mac OS X)\n\n  Unpack the tarball in the desired location (e.g. under `~/astrotools/`):\n\n\t\u003e cd ~/astrotools\n\t\u003e tar xzf crush-2.xx-x.tar.gz\n\n  Verify that CRUSH works:\n\n\t\u003e cd crush\n\t\u003e ./crush\n\n  You should see a brief information screen on how to use CRUSH.\n\n\n\n#### B. Installation via Linux packages\n  \n  Use the package manager on your system to install. E.g., on RPM-based \n  systems (Fedora, RHEL, CentOS, SUSE) you may type:\n\n\t\u003e sudo yum localinstall --nogpg crush-\u003cversion\u003e-noarch.rpm\n\n  On Debian based systems (e.g. Ubuntu, Mint) the same is achieved by:\n\n\t\u003e sudo apt-get install crush-\u003cversion\u003e.deb\n\n  Or, you may simply double click the package icon from a file browser. \n\n\n\n#### C. Windows Installation\n   \n  Please refer to `README.windows`, which may be found in the \n  `Install\\windows\\` subdirectory of the distribution, or online under the \n  'Documents' tab.\n\n \n\n### 1.2. (optional) Java Configuration\n\n\n CRUSH ships with a default Java configuration. On Windows and the most \n common POSIX platforms (Linux, Mac OS X, BSD, and Solaris), it will \n automatically attempt to set an optimal configuration. On other platforms, \n it comes with fail-safe default values (default java, 32-bit mode and 1GB \n of RAM use).\n\n You can override the defaults by placing your settings in arbitrary files \n under /etc/crush2/startup or `~/.crush2/startup` (for the equivalent \n configuration under Windows, please refer to `README.windows`)\n\n (Any settings in the user's home under `.crush2/startup` will overrride the \n system-wide values in `/etc/crush2/startup` or `C:\\Program Data\\startup`. \n If multiple config files exist in the same location, these will be parsed in \n non-specific order). \n\n E.g., placing the following lines in `~/.crush2/startup/java.conf` overrides \n all available runtime configuration settings:\n\n    JAVA=\"/usr/java/latest/bin/java\"\n    USEMB=\"4000\"\n    JVM=\"-server\"\n    EXTRAOPTS=\"-Djava.awt.headless=true\"\n\n upon startup CRUSH will find and apply these settings, so it will use\n `/usr/java/latest/bin/java` to run CRUSH with 4GB of RAM, using the HotSpot \n `server` VM, and in headless mode (without display, mouse or keyboard).\n\n Below is a guide to the variables that you can override to set your own\n Java runtime configuration:\n\n    JAVA       Set to the location of the Java executable you want to use. \n               E.g. \"java\" to use the default Java, or\n               `/usr/java/latest/bin/java` to use the latest from Oracle or \n               OpenJDK.\n\n    USEMB      Set to the maximum amount of RAM (in MB) available to \n               CRUSH. E.g. \"4000\" for 4GB. On a 32-bit OS, or with a 32-bit\n               Java installation no more than 2GB of RAM may be accesssed. In\n               practice, the maximum 32-bit OS/Java values range from around \n               \"1000\" (32-bit Windows / Java) to \"1900\" (32-bit Linux / Java).  \n\n    JVM        Usually set to `-server` for Oracle or OpenJDK. If using IBM's \n               Java, set it to \"\" (empty string). On ARM platforms, you \n               probably get better performance using `-jamvm` or `-avian`. To \n               see what VM options are available on your platform, run \n               `java -help`. The VM options are listed near the top of the \n               resulting help screen. \n\n    EXTRAOPTS  Any other non-standard options you may want to pass to the Java \n               VM should go here. \n\t\n\n You can also specify environment variables, and add shell scripts (bash),\n since these configuation files are in fact sourced as bash scripts before\n launching Java / CRUSH. For example you can add:\n\n```bash\n CRUSH_NO_UPDATE_CHECK=\"1\"\n CRUSH_NO_VM_CHECK=\"1\" \n\t  \n echo \"Will try to parse my own configuration now... \"\n\n if [ -f \\~/mycrushconfig.sh ] ; then\n   echo -n \"OK\"\n   source \\~/mycrushconfig.sh\n else\n   echo -n \"Not found\"\n fi\n```\n\n The above will disable update checking (not recommended!) and VM checking\n (also not recommended!) and will source the contents of \n `~/mycrushconfig.sh` if and when such a file exists.\n\n\n\n### 1.3.  CRUSH Pipeline Configuration\n\n The preferred way of creating user-specific configurations for CRUSH is to\n place your personalized configuration files inside a `.crush2/` \n configuration directory within your home. This way, your personalized \n configurations will survive future updates to CRUSH.\n\n You can create/edit your default configuration by editing `default.cfg`\n (either in the installation folder or `~/.crush2`, or in the appropriate\n instrument subdirectory within it. \n As an example a user configuration for LABOCA data can be placed into \n `~/.crush2/laboca/default.cfg`, with the a content:\n\t\n\tdatapath \\~/data\n\toutpath \\~/images\n\tproject T-79.F-0002-2006\n\n This tells crush that when reducing LABOCA data it should look for raw data \n files in `~/data`, write all output to `~/images`, and specifies the default\n project to be `T-79.F-0002-2006`\n \n The tilde character `~` is used as a shorthand for the home directory \n (similarly to UNIX shell syntax). In your configuration you can also refer \n to environment variables or other settings (see more about it further below).\n\n Of course, you can create any number of configurations, name them as you \n like place them where you like (practical if you have many data locations, \n projects etc.). You can easily invoke configuration files as needed via\n\n\t\u003e crush [...] -config=\u003cpath-to-myconfig\u003e [...]\n\n\n\n\n### 1.4. The Tools of the CRUSH Suite\n\n CRUSH provides a collection of other useful tools. Here's a short description\n of what is there and what they are used for. Each tool, when run without\n options (or with the -help option) will provide you a list of available\n options on the console.\n\n    crush       The principal reduction tool.\n\n    imagetool   A tool for manipulating images. Can also deal with \n                images produced by BoA (and to some degree other \n                images also).\n\n    show        A simple display program for the FITS files, with \n                various useful functions for simple analysis and image\n                manipulation capabilities.\n\n    coadd       Add FITS images together. Use as a last resort tool as\n                it is always better to reduced scans together.\n\n    difftool    Allows to look at the difference between two images.\n\t\t(used to be named 'difference')\n\n    histogram   Write a histogram of the pixel distribution of an \n                image plane. (e.g. `flux`, `rms`, `s2n`, `weight`, or \n                `time`).\n\t\n    detect      A source extraction tool for maps. You should make\n                sure that the noise is close enuogh to Gaussian (e.g. \n                with `histogram`) before relying on this. \n\n    esorename\t Rename ESO archive files to their original names.\n\n\n\n\n### 1.5. Quick Start\n\n To reduce data, simply specify the instrument (E.g. `sharc2`, `laboca`...) \n and the scan numbers/ids (or names of the scan files or directories) to \n crush. (You may have to add `./` before `crush` if the current directory \n `.` is not in your path.) E.g.,\n\n\t\u003e crush laboca 10059\n\n will reduce LABOCA scan 10059 with the default reduction parameters.\n (Try `crush -help` to see a full list of the supported instrument names.)\n\n After the instrument name, you can specify additional options. Some of these\n apply to the reduction as a whole while others will affect the scan \n processing for those scan numbers that are listed *after* the option flag. \n\n If you are new to CRUSH (or used version 1.xx before), you should be able\n to start working with it by the time you get to the end of this page.\n Nevertheless, I recommend that you read on through the entire Sections \n 1--2 (Getting Started \u0026 Basic Configuration), and you will become a\n truly well-versed user. :-)\n\n Once you get the hang of it, and feel like you need more tweaking ability,\n feel free to read on yet further to see what other fine tuning capabilities\n exist...\n\n Here are some quick tips:\n\n * Reduce as many scans together as you can. E.g.\n\t\t\n        \u003e crush laboca 10657-10663 10781 11233-11235 [...]\n\n * You can specify opacities, pointings, scaling etc, for each\n   of the set of scans listed (See section for Scan-specific options). \n   E.g.,\n\n        \u003e crush laboca -tau=0.32 10657-10663 10781 -tau=0.18 11233 [...]\n\n   will use a zenith tau value of 0.32 for 10657-10663 and 10781, and\n   0.18 for the last scan (11233).\n\n * If you suspect that you are missing extended emission (see section\n   on the Recovery of Extended Structures), then you can specify the\n   'extended' option, which will better preserve large scale structures\n   albeit at the price of more noise. E.g.\n\n        \u003e crush laboca -extended 10657-10663\n\n * If your source is faint (meaning S/N `\u003c` 10 in a single scan), then\n   you may try using the `faint` option. E.g.\n\n        \u003e crush laboca -faint 10657-10663\n\n   or,\n\n        \u003e crush laboca -faint -extended 10657-10663\n\n   to also try preserve extended structures (see above).\n\n * For finer control of how much large scale structures are retained\n   you can use the `sourcesize` option, providing a typical source \n   scale (in arcsecs) that you'd like to see preserved. Then CRUSH will\n   optimize its settings to do the best it can to get clean maps while \n   keeping structures up to the specified scales more or less intact. \n   E.g.\n\n        \u003e crush [...] -sourcesize=45.0 10657-10663\n  \n   Will optimize reduction for \u003c\\~ 45 arcsec sources.\n\n * Finally, if your sources are not only faint, but also point-like, \n   you should try the `deep` option. This will use the most aggressive\n   filtering to yield the cleanest looking maps possible.\n   E.g.,\n\n        \u003e crush laboca -deep 10657-10663\n\n   With just these few tips you should be able to achieve a decent job in\n   getting the results you seek. CRUSH can also help you reduce \n   pointing/calibration, skydip, and pixelmap scans. E.g.:\n\n * To reduce a laboca pointing/calibration scan 11564:\n\n        \u003e crush laboca -point 11564\n\n   At the end of the reduction CRUSH will suggest pointing corrections\n   and provide detailed information on the source (such as peak and\n   integrated fluxes, FWHM and elongation). Once you determine the\n   appropriate pointing corrections for your science scans, you\n   can apply these via the `pointing=x,y` option. E.g.:\n\n        \u003e crush laboca -pointing=3.2,-0.6 12069\n\n * You can also reduce skydips, for determining in-band atmospheric\n   opacities. E.g.:\n\n        \u003e crush hawc+ -skydip 26648\n\t  \n * Finally, you can derive pixel position information from appropriate\n   scans, which are designed to ensure the source is moved over every\n   detector, in a fully sampled manner. To reduce such beam-maps:\n\n        \u003e crush gismo -pixelmap 5707\n\n   CRUSH will write rcp files as output, containing pixel positions\n   source and sky gains in a standard 5 column APEX RCP format. \n   You can feed the pixel position information to reduce other scans \n   via the `rcp` option.\n\n   \n There are a lot more fine-tuning possibilities for the more adventurous. If \n interested, you can find documentation further below, as well as in the \n README files inside the instrument sub-directories. For a complete list of \n crush options, please refer to the GLOSSARY.\n\n\n   \n\n### 1.6. A Brief Description of What CRUSH Does and How...\n\n CRUSH is a pipeline reduction that is principally meant to remove\n correlated signals (correlated noise) in the time-streams to arrive\n at clean \u0026 independent bolometer signals, which are then used to make\n a source model (usually an image).\n\n As such it is not an interactive reduction software (e.g. as opposed to \n e.g. BoA). The term 'scripting' in CRUSH mainly means defining configuration \n options (in the command line or through configuration files) which are \n parsed in the order they are read.\n\n During the reduction CRUSH aims to arrive at a consistent set of\n solutions for various correlated signals, the corresponding gains and\n the pixel weights, as well as tries to identify and flag problematic data.\n\n This means a series of reduction steps, which are iterated a few times\n until the required self-consistent solutions are arrived at. \n   \n To learn more about the details please refer to Kovacs, A., \"CRUSH: \n fast and scalable data reduction for imaging arrays,\" Proc. SPIE 7020, 45, \n (2008). If that does not satisfy your curiousity, then you can find yet\n more explanation in Kovacs, A., PhD thesis, Caltech (2006).\n\n\n\n\n### 1.7. Command-line Options and Scripting Support\n\n Configuration of CRUSH is available either through command line \n options or via scripting. You have seen scripting already in the form\n of 'default.cfg', which stores the default configuration values (Sec. 1.1). \n\n The syntax is designed so that it accomodates both scripting and\n command-line (bash) use alike. Thus, white spaces are optional and curved \n brackets are avoided (unless these are placed in quotes).\n\n For a complete guide on using the configuration engine, please read\n README.syntax. Below we provide only a very basic overview, and describe\n only what is specific to CRUSH, beyond the cofiguration engine in general.\n\n\n#### Basic Rules\n \n  When defining settings, keys can be separated from their values either by\n  `=`, `:` or empty spaces (or even a combination of these).\n\n  Command line options start with a dash `-` in front. Thus, what may look\n  like:\n\t\n\tkey1 value1\n\tkey2 value2, value3\n\n  in a configuration script, will end up as\n\n\t\u003e crush [...] -key1=value1 -key2=value2,value3 [...] \n\n  on the command line. Otherwise, they two ways of configuring are generally \n  equivalent to one-another. One exception to this rule is reading scans,\n  which is done via the `read` key in a script, but on command line you can\n  simply list the scan number (or ranges or lists or names). I.e., \n\n\t[...]\n\tread 10056                # in script\n\n\t\u003e crush [...] 10056       # on the command line.\n\n  In the next section you'll find a description of the scripting keywords.\n  Now that you know how to use them also as command line options, you\n  can choose scripting or command-line, or mix-and-match them to your\n  liking and convenience...\n \n  Key/value pairs are parsed in the order they are specified. Thus, each\n  specification may override previously defined values. \n\n  Lines that start with `#` designate comments that are ignored by the \n  parser.\n \n\n \n#### Dynamic Conditionals\n \n Some conditions are interpreted dynamically. For example CRUSH, can \n activate settings based on observation date (or MJD) or serial number, on \n a scan-by-scan basis, such as:\n\n\tmjd.[54555-54869] rcp {$CRUSH}/laboca/laboca-3.rcp\n \n which loads the pixel positions (RCP) `laboca-3.rcp` inside the `laboca`\n subfolder in the crush installation for scans taken in the MJD range\n 54555 to 54869. Similarly,\n\n\tdate.[2007.02.28-2009.10.13] instrument.gain -1250.0\n\n or\n\n\tserial.[14203-15652] flag 112-154,173\n\n are examples of setting activated based on dates or scan serial numbers.\n\n You may also specify conditional settings based on the\n source names under the `object` branch. E.g.:\n\n\tobject.[Jupiter] bright\n\n will invoke the `bright` configuration for Jupiter. The check for the\n source name is case insensitive, and matches all source names that begin\n with the specified sequence of characters. Thus, the SHARC-2 configuration\n line:\n\n\tobject.[PNT_] point\n\n will perform a pointing fit at the end of the reduction for all sources\n whose catalog names begin with `PNT_` (e.g. `PNT_3C345`).\n\n These examples also demonstrates that conditionals can be branched just\n like options. (In the above cases, the conditions effectively reside in the\n `mjd`, `date` or `serial` branches). Other commonly used conditionals of\n this type are iteration based settings:\n\n\t\u003e crush [...] -iteration.[last-1]whiten=2.0 [...]\n\n will activate the noise whitening in the iteration before the last one.\n\n   \n#### Startup Configuration\n   \n  At launch, crush will parse `config/default.cfg` to establish a default \n  configuration set. All configurations files are first parsed from the \n  files in the `config/` directory inside crush, then additional options or \n  overriders  are read also from the appropriate instrument subdirectories, \n  if exist.\n \n  After that, crush will check if these configuration files also exists \n  under the inside the `~/.crush2` directory of the user. If so, they will be\n  used to override again. See more on this in the next section under the\n  `config` option. In this way, the `~/.crush` directory can be used as\n  a convenient way to create user specific setting and/or overrides to\n  global defaults.\n\n\n\n\n\n### 1.8. Making sense of the console output\n \n Don't ignore the console output. It contains a lot of information, which\n may be useful for diagnosing your reduction, and troubleshooting if things\n don't go as expected. \n\n In the beginning of the console output, you will see information from each\n scan as it is being read, followed by output from the preprocessing steps\n (such as downsampling, scan statistics, velocity clipping, opacity used,\n pointing corrections applied etc.). This part is verbose enough that it \n should be eszsentially self-explanatory.\n\n Once all the scans are loaded, and the source model (usually a map) has been\n created to accomidate the dataset, CRUSH will enter the iterated pipeline\n reduction. It may look cryptic at first, as it captures a lot of information\n in very little console space. Below is a short guide to figure out what\n it all means. Understanding it will help you diagnose and troubleshoot all\n sorts of issues that might affect the reduction of your dataset. So, don't\n be shy, dive in!\n\n\n##### An example line from the console output\n\n Let's begin with a concrete example. Here is a section from the console \n output from the reduction of LABOCA scan 11564 (from 2007): \n\n    $ Round 4: \n    $\n    $  [11564] D(128) C245 W(8.44E-2)245 dN(0)245 B245 c245 Map(8.44E-2) \n    $  [Source] {check} (smooth) (clip:4.0) (blank:10.0) (sync) \n\n Each _phrase_ (separated by spaces around it) represents a short summary of\n a pipeline step. The leading characters identify the step, often followed by \n some quantitative capure (or figure-of-merit) summarizing what the step\n accomplished.\n\t\n Here is how the above section is interpreted: \n\n   In the 4th iteration, the following steps are performed on scan 11564: \n\n    D(128)        1/f filtering (on 128 frame timescale)\n\n    C245          removing the correlated noise from the array with \n                  245 pixels surviving the gain range criterion.\n\n    W(8.44E0-2)   Deriving pixel weights with the `rms` method. The\n                  average sensitivity of the array is estimated to be\n                  84.4 mJy sqrt(s).\n\n    dN(0)245      Despiking via the `neighbours` method, with 0 frames\n                  flagged as spiky, and 245 pixels surviving the\n                  flagging of pixels with too many spikes.\n\n    B245          Decorrelating on amplifier boxes, with 245 pixels \n                  having acceptable gains to the correlated signals.\n\n    c245          Decorrelating on electronic cables, with 245 pixels\n                  showing acceptable gain response to these signals.\n\n    Map(8.44E-2)  Mapping scan, with estimated point source NEFD of\n                  84.4 mJy sqry(s).\n\n   Then, at the end of the iteration a composite source model is created. This\n   is further processed as:\n\n    {check}       Remove invalid data from the scan maps (before adding\n                  to composite).\n\n    (smooth)      smooth composite by the desired amount.\n\n    (clip:4.0)    Discard map points below an S/N of 4.0 from composite.\n\n    (blank:10.0)  Do not use bolometer data in other steps, which are\n                  collected over the map points with S/N \u003e 10, thus\n                  reducing the bias that bright sources can cause in the\n                  reduction steps.\n\n    (sync)        Synchronize the composite model back to the time-stream\n                  data.\n\n Once the reduction is done, various files, including the source map, are\n written. This is appropriately echoed on the console output.\n\n\n   \n#### Console Output Reference Guide\n\n  Below is a reference guide for interpreting the console output for your \n  particular reduction:\n  \n\n##### Quantities inside the brackets:\n\n    [nnnnn|m]:  processing scan nnnnn, subscan m from the list\n\n    (#.##E#)    The effective point-source NEFD (usually after \n                weighting or at the mapping step)\n                shown in the effective mapping units times sqrt(s).\n\n    []          bracketed models are solved via median estimators\n\n    (#)         Indicates the time resolution of the given step as\n                the number of downsampled frames, when this is not\n                the default full resolution.\n\n\n##### Reduction-step Shorthands (alphabetically)\n\n    a          Decorrelating amplifier boards (e.g. LABOCA,GISMO).\n\n    am         Remove correlated telescope acceleration (magnitude)\n\n    ax         Remove correlated telescope x acceleration\n\n    ay         Remove correlated telescope y acceleration\n\n    B          Decorrelating electronic boxes (e.g. LABOCA).\n\n    C          Solving for correlated noise and pixel gains\n               The time resolution (in frames) is shown in brackets\n               if not full resolution, followed by the number of \n               unflagged pixels if gains are solved and a gain range\n               is defined for flagging.\n\n    c          Decorrelating electronic cables (e.g. LABOCA)\t\n               or geometric columns (e.g. GISMO)\n\n    cx         Remove correlated chopper x position\n\n    cy         Remove correlated chopper y position\n\n    D          Solving for pixel drifts (1/f filtering).\n\n    dA(#)      despiking absolute deviations.\t\n               In brackets it shows the percentage # of frames\n               flagged in the data as spiky by any method.\n\n    dG(#)      like above but proceeds gradually.\n\n    dN(#)      despiking using neighbouring samples in the timeseries.\n\n    dM(#)      despiking at multiple resolutions (up to the 1/f\n               filter timescale).\n\n    E          Remove correlated SAE error signal (GISMO)\t\n\n    G          Estimating atmospheric gradients accross the FOV.\n\n    J          De-jumping frames (e.g. for GISMO). Followed by two\n               colon (:) separated numbers: the first, the number of\n               de-jumped blocks that were re-levelled and kept; and\n               the number of blocks flagged.\n\n    m          Decorrelating on multiplexers (e.g. GISMO, SCUBA-2)\n\n    Mf         Filtering principal telescope motion.\n\n    O          Solving for pixel offsets\n\n    p          Decorrelating on (virtual) amplifier pins (e.g. GISMO)\n\n    Q          Decorrelating wafers (e.g. ASZCA).\n\n    r          Decorrelating on geometric rows (e.g. SHARC-2, GISMO)\n\n    t          Solving for the twisting of band cables (e.g. LABOCA).\n\n    tx         Remove correlated telescope x position\n\n    ty         Remove correlated telescope y position\n\n    tW         Estimating time weights.\n\n    W          Estimating pixel weights.\n\n    w          Estimating pixel weights using the 'differential'\n               method.\n\n    wh(x.xx)  Noise whitening. The average source throughput factor\n              from the whitening is shown.\t\n\n##### Source model-specific:\n\n     \n    Map         Calculating source map from scan. The effective point\n                source NEV/NEFD of the instrument is shown in the \n                brackets (e.g. as Jy sqrt(s)).\n\n    [C1\\~#.##]  Filtering corrections are applied directly and are\n                #.## on average.\n\n    [C2=#.##]   Faint structures are corrected for filtering after\n                the mapping via an average correction factor #.##.\n\n    [Source]    Solving for source map.\n\n    {check}     Discarding invalid map data.\n\n    {despike}   despiking scan maps before adding to composite.\n\n    {filter}    Spatial large-scale structure (LSS) filtering of the \n                scan maps.\n\n    (level)     Zero levelling to map median.\n\n    (smooth)    Smoothing map.\n\n    (filter)    Spatial filtering of the composite map.\n\n    (noiseclip) Clip out the excessively noisy parts of the map.\n\n    (filter)    Filtering large scale structures (i.e. sky-noise).\n\n    (exposureclip)  Clipping under-exposed map points.\n\n    (blank)    Blank excessively bright points from noise models.\n\n    (sync)     Removing the source model from the time-stream.\n\n\n\n\n\n### 1.9. Examples\n\n Reduce scans 12065-12069 and 12072 with zenith tau of 0.18:\n\n\t\u003e crush laboca -tau=0.18 12065-12069 12072\n\n Reduce scans 10562 and 10645 together, with the first scan observed at a\n zenith tau of 0.21, and the second at tau of 0.35 with.\n\n\t\u003e crush laboca -tau=0.21 10562 -tau=0.35 10645\n\n Say you realize that the pointing was off by -5.4\" in AZ, and 2.4\" in EL\n for the second scan. Then:\n\n\t\u003e crush laboca -tau=0.21 10562 -tau=0.35 -pointing=-5.4,2.4 10645\n\n Say scan 10049 is a scan of a bright source (e.g. Saturn) and the default\n reduction ends up clipping much of it. Then,\n\n\t\u003e crush laboca -bright 10049\n\n If the source still gets nipped from the resulting map, you can try \n disabling pixel weigting altogether (this is the likely culprit) by:\n\t\n\t\u003e crush laboca -bright -blacklist=weighting 10049\n\n Perhaps you suspect there is missing extended emission (large-scale \n structure) in scan 10550. You can try to recover more by:\n\n\t\u003e crush laboca -extended 10550.\n\n Try reduce some faint source (scan 10057):\n\n\t\u003e crush laboca -faint 10057\n\n Maybe your faint source has extended structure that you want to try recover\n better, and willing to pay some noise penalty for it. Then try:\n\n\t\u003e crush laboca -faint -extended 10057.\n\n You can also fine-tune, what is the largest source-scale (more-or-less) \n that you are interested in. Then the reduction will adjust its parameters\n accordingly. Say you expect your source to be a blob around 1' in diameter,\n then you can try:\n\n\t\u003e crush laboca -faint -sourcesize=60.0 10057.\n\n You can also play with the blanking clipping methods above, if there is\n annoying negative dips remaining around your brighter peaks. Note, that\n you porbably want to stick with -extended, or else such dips may be the\n result of the aggressive filtering settings applied to your specific\n source size (via `sourcesize`).\n\n As mentioned before, command-line options and scripting are equivalent\n ways of configuring the reduction. Thus, the running the script `test.cfg`:\n\n\tfaint\n\textended\n\ttau 0.18\n\tpointing -3.2,4.8\n\tread 12065-12069\n\tpointing 2.3,-0.5\n\tread 12072\n\tblank 10.0\n\tclip 3.0\n\titeration.[last-1] forget clip\n\n via \n\n\t\u003e crush laboca -config=test.cfg\t\n\n is equivalent to the command line:\n\n\t\u003e crush laboca -faint -extended -tau=0.18 \\\n\t  \t-pointing=-3.2,4.8 12065-12069 \\\n\t\t-pointing=2.3,-0.5 12072 \\ \n\t\t-blank=10.0 -clip=3.0 -iteration.[last-1]forget:clip  \n\n\n Finally, suppose you want to reduce a dataset on a very faint point source, \n such as a distance galaxy, or a faint core, with a set of scans:\n\n\t\u003e crush laboca -deep 23165-23169 \n\n And, perhaps you want to comprate the result to a random jackknife, which\n you can obtain with:\n\n\t\u003e crush labova -deep -jackknife 23165-23169\n   \n\n\n\n----------------------------------------------------------------------------\n\n## 2. Advanced Topics\n\n\n\n### 2.1. Pointing Corrections\n\n Reducing the data with the correct pointing can be crucial (esp. when \n attempting to detect/calibrate faint point sources). At times the pointing\n may be badly determined at the time of observation. Luckily, getting the \n exact pointing offset wrong at the time of the observation has no serious \n consequences as long as the source still falls on the array, and as long as \n the exact pointing can be determined later. If you, at the time of the data \n reduction, have a better guess of what the pointing was at the time the\n data was taken (e.g. by fitting a model to the night's pointing data), you \n can specify the pointing offsets that you believe were more characteristic\n to the data, by using the `-pointing` option *before* listing the scans \n that should be reduced with the given corrections. E.g.,\n\n\t\u003e crush [...] -pointing=12.5,-7.3 \u003cscans\u003e\n\n Will instruct minicrush that the _true_ pointing was at dAZ=12.5 and \n dEL=-7.3 arcsec each (i.e. it works just like pcorr on APECS).\n\n Some instruments, like SHARC-2, may allow specifying the aggregated pointing\n offsets (e.g. `fazo` and `fzao`) instead of the differential corrections \n supplied by `pointing`.\n\n\n#### Obtaining Corrections\n\n  Good practice demands that you regularly observe pointing/calibration \n  sources near your science targets, from which you can derive appropriate\n  corrections. CRUSH provides the means for a analyzing pointing/calibration\n  data effectively, using the `point` option:\n\n\t\u003e crush [...] -point [...]\n\n  At the end of the reduction, CRUSH will analyze the map, and suggest\n  appropriate pointing corrections (to use with `-pointing`, or other\n  instrument-specific options), and provide calibration information as well\n  as some basic measures of source extent and shape.\n\n\n\n#### After Reduction (a poor alternative)\n   \n  You can also make pointing changes after the reduction (alas, now in \n  RA/DEC). You can read off the apparent source position from each separately \n  reduced scan (e.g. by using `show` and placing the cursor above the apparent\n  source center/peak). Then you can use `imagetool` to adjust the pointing. \n  E.g.:\n\n\t\u003e imagetool [...] -origin=3.0,-4.5 ... \n\n The above moves the map origin by 3\" in RA and -4.5\" in DEC. \n\n Then, other crush tools (like `coadd`, `imagetool` etc.) will use these images \n with the proper alignment. Clearly, this method of adjusting the pointing is\n only practical if your source is clearly detected in the map.\n\n\n\n\n### 2.2. Pixelization and Smoothing\n\n There seem to be many misconceptions about the _correct_ choice of \n pixelization and about the (mal)effects of smoothing. This section aims to \n offer a balanced account on choosing an appropriate mapping grid and on the \n pros and cons of applying smoothing to your maps.\n\n#### Pixelization (i.e. choosing the 'right' grid)\n\n  **Misconception**: You should always _Nyquist_ sample your map, with 2 \n  pixels per beam FWHM, to ensure ideal representation.\n\n  There is more than one thing wrong with the above idea. First, there is no\n  'Nyquist' sampling of Gaussian beams. The Nyquist sampling theorem applies \n  only to data, which has a natural cutoff frequency (the Nyquist frequency), \n  above which there is no information present. Then, and only then, will \n  sampled data (at a frequency strictly larger[!] than twice the Nyquist \n  cutoff) preserve all information about the signals. \n\n  Two pixels per beam is almost certainly too few for the case of Gaussian \n  beams. Gaussian beams have no true frequency cutoff -- the signal spreads\n  across the spectrum, with information present at all frequencies, even if it\n  is increasingly tapered at the higher frequencies. By choosing a sampling \n  (i.e. pixel size in your map) the information above your sampling rate will \n  be aliased into the sampled band, corrupting your pixelized data. \n\n  Thus, you can choose a practical cutoff frequency (i.e. pixelization) based\n  on the level of information loss and corruption you are willing to tolerate.\n  At 2.5 pixels pear FWHM, you retain \\~95% of the underlying information, and \n  corrupt it by the remaining 5%. With more pixels per beam, you get more \n  accurate maps. (The 2-pixels per beam, that many understand to be the \n  Nyquist  sampled, is certainly too few by most standards of accuracy!)\n\n  Thus, the CRUSH default is to use around 5-pixels per FWHM, representing a \n  compromise between completeness (\\~99%) with minimal corruption (`\u003c`1%) and a \n  senseless increase of map pixels (i.e. number of model parameters).\n\n\n#### Smoothing\n   \n  **Misconception**: You should not smooth your maps, because it does \n  something _unnatural_ to your _raw_ image.\n\n  The first problem with this view is that there is no _natural_ or _raw_ image\n  to start with, because there is no _natural_ pixelization (see above). \n  Secondly, by choosing a map pixel size, you effectively apply smoothing -- \n  only that your smoothing kernel is a square pixel, with an abrupt edge, \n  rather than a controlled (and directionless) taper of choice. \n\n  (To convice yourself that a map pixels apply smoothing, consider the mapping \n  process: each pixel will represent an average of the emission from the area \n  it covers. Thus, the pixel values are essentially samples of a moving \n  integral under the pixel shape -- i.e. samples of the emission convolved \n  [smoothed] by the pixel shape.)\n\n  However, smoothing (including coarse pixelization!) does have a real \n  downside, in that it degrades the effective resolution of the image. Consider\n  smoothing by Gaussian kernels. Then, the image resolution (imageFWHM) \n  increases with the smoothing width (`smoothFWHM`) as\n\n\timageFWHM^2 = beamFWHM^2 + smoothFWHM^2\n\n  from the instrument resolution (`beamFWHM`). However, you can smooth a fair \n  bit before the degradation of resolution becomes a problem or even really\n  noticeable. If you use sub-beam smoothing with `smoothFWHM \u003c beamFWHM`, then \n  the relative widening is roughly\n\n\tdW / W0 ~ 0.5 * (smoothFWHM / beamFWHM)^2\n\n  Thus, smoothing by half a beam, degrades resolution by \\~12% only... At the \n  same time, smoothing can bring many benefits:\n\n * Improves image appearance and S/N by rejecting pixelization noise. It is\n   better to use a finer grid and smooth the image by the desired amount,\n   than to pixelize coarsely -- not only for visual appearance but also\n   for the preservation of information.\n\n * Smoothing can help avoid pathological solutions during the iterations, \n   by linking nearby pixel values together.\n\n * Beam-smoothing is the optimal (Wiener) filtering for determining point \n   source fluxes in a white noise background. I.e., smoothing by the \n   beam produces the highest signal-to-noise measurement of point sources.\n\n * Beam-smoothing is mathematically equivalent to fitting fixed-width beams\n   at every pixel position. The beam-smoothed flux value in each pixel\n   represents the amplitude of a beam fitted at that position using \n   chi-squared minimization. Thus, beam smoothed images are the bread-and-\n   butter of point source extraction (e.g. the `detect` tool of CRUSH).\n\n  Given the pros and the con for smoothing, the different reduction modes \n  (_default_, `faint` or `deep`) of CRUSH make different compromises. The \n  default reduction aims to provide maps with maximal resolution (i.e. no \n  smoothing), although a some smoothing is used during the iterations to aid\n  convergence to a robust solution. `faint` mode reductions smooth by 2/3 \n  beams (resulting in \\~22% degradation of resolution), to provide better \n  signal-to-noise at a moderate loss of spatial resolution. \n  Finally, `deep` reductions yield beam smoothed images, which are ideal for \n  point source extraction, even if some spatial details are sacrificed.\n\n  You can change/override these default settings. The smoothing during \n  reduction is  controlled by the `smooth` setting, which can take both a \n  Gaussian FWHM (in arcsec) as its arguments as well as the preset values \n  `minimal`, `halfbeam`, `2/3beam` and `beam`. The smoothing of the final map \n  can be controlled by 'final:smooth' (the `final` stands as a shorthand \n  conditional for the last iteration). Thus,\n\n\t\u003e crush [...] -smooth=halfbeam -final:smooth=minimal [...]\n\n  smoothes by half a beam during the iterations, and only slightly for the \n  output image, while\n\n\t\u003e crush [...] -forget=smooth -final:forget=smooth [...]\n\n  disables smoothing both during reduction and at the end. The table below\n  summarizes the effect of the preset smoothing values, indicating both the\n  degradation of resolution (relative to the telescope beam) and the relative\n  peak S/N on point sources:\n\n\n   \n   **Table 1.** _Smoothing properties_\n\n   _smooth_       | widening     |\trel. S/N\n   ---------------|--------------|--------------\n   minimal        |      6%      |      0.33\n   halfbeam       |      12%     |      0.50\n   2/3beam        | \t 22%     |      0.67\n   beam           |      41%     |      1.00\n\n\n\n\n\n\n### 2.3. Recovery of Extended Emission\n\n As a general rule, ground based instruments (in the mm and submm) are only \n sensitive to structures smaller than the Field-of-View (FoV) of the \n instrument. All scales larger than the FoV will be strongly degenerate with \n the bright correlated atmosphere (a.k.a. sky noise), and will be very \n difficult, if not outright impossible, to measure accurately. In a sense, \n this is analogous to the limitations of interferometric imaging, or the \n diffraction-limited resolution of finite apertures. \n\n However, there is a some room for pushing this limit, just like it is \n possible to obtain some limited amount of super-resolution (beyond the \n diffraction limit) using deconvolution. The limits to recovery of scales \n over the FoV is similar to those of obtaining super-resolution beyond the \n diffraction limit. Both these methods\n\n 1. yield imperfect answers, because the relevant spatial scales are \n    poorly measured in the first place.\n\n 2. are only feasible for high signal-to-noise structures.\n\n 3. can, at best, go beyond by a factor of a few beyond the fundamental \n    limit.\n\n You can usually choose to recover more extended emission if you are willing \n to put up with more noise on those larger scales. This trade-off works \n backwards too -- i.e., you can get cleaner maps if you are willing to filter\n them more.\n\n As a general principle, structures that are bright (`\u003e\u003e` 5-sigma) can be fully\n recovered to up to a few times the field-of-view (FOV) of the bolometer \n array. However, the fainter the structure, the more it will be affected by \n filtering.\n\n Generally, the fainter the reduction mode, the more filtering of faint\n structures results, and the more limited the possibility of recovering \n extended structures becomes. The table below is a rough guide of what \n maximum scales you may expect for such faint feaures, and also, how noise is\n expected to increase on the large scales as these are added in `extended` \n mode:\n\n\n   **Table 2.** _Maximum faint structure scales for S/N `\u003c`\\~ 5_\n\t\n   option              | Max. sensitive scale        | Noise power scaling \n   --------------------|-----------------------------|--------------------\n   _default_ / `bright`|          FOV/2              |  \\~L^2\n   `+ extended`        |           FOV               |  \\~L^2\n   `faint` / `deep`    | `2*sourceSize` or `2*beam`  |   \\~L\n   `+ extended`        |          FOV/2              |   \\~L\n\n\n Iterating longer, will generally help recover more of the not-too-faint \n large-scale emission (along with the large-scale noise!). Thus,\n\n   \t\u003e crush -extended -rounds=50 [...] \n\n will generally recover more extended emission than just the default\n\n   \t\u003e crush -extended [...]\n\n (which iterates 10 times). In general, you may expect to recover significant\n (`\u003e`5 sigma) emission up to scales L as:\n\n\tL ~= FoV + sqrt(N v T)\n\n in terms the number of iteration N, limiting Field-of-View (FoV), scanning \n velocity v and correlated noise stability time-scale T. Unfortunately, the \n correlated noise stability of most ground-based instruments is on the order \n of a second or less due to a highly variable atmosphere. At the same time, \n the noise rms on the large scales will increase assymptotically as:\n\n\trms ~ sqrt(N)\n\n for large numbers of iterations.\n\n\n\n\n\n### 2.4. Filtering corrections, transfer functions, etc.\n\n Most reduction steps (decorrelation, 1/f drift removal, spectral filtering) \n will impact different spatial frequencies of the source structure \n differently. In general, the low spatial frequency components are most \n affected (suppressed). E.g. decorrelating the full array will reject \n scales larger than the field-of-view, while 1/f filtering will result in\n 1/f spatial filtering of the map due to the scanning motion, which more\n or less directly maps temporal frequencies into spatial frequencies.\n\n The approach of CRUSH is to apply appropriate point-source corrections\n (rescaling) such that point sources in the map yield the same fluxes no\n matter how (i.e. with what options exactly) the source was reduced. \n While the corrections will be readily sufficient for a large fraction of \n the science cases in this simple form, there are two notable exceptions\n to this rule: (i) extended emission, and (ii) small, fast maps reduced\n in `deep` mode (when the source is scanned over the same detector more than\n once over the `stability` timescale). \n\n The better approach for these cases is to measure a transfer function, or \n otherwise check the reduction of a similar simulated source.\n\n\n#### Transfer functions and simulated sources\n\n  The `sources` option provides a means for inserting test sources into\n  CRUSH maps, while one of the `jackknife` options can be used to remove\n  any real emission beforehand but retaining the signal and noise structure\n  of the data otherwise (which is important in order to get a representative\n  measure of the transfer function). E.g.\n\n\t\u003e crush [...] -jackknife.alternate -sources=test.mask ...\n\n  will apply an alternating jackknife to the input scans, and insert sources\n  specified in the mask file `test.mask` (See `example.mask` on the format\n  of mask files, and the GLOSSARY for more on jackknifing options).\n\n  To measure transfer functions (i.e. complete spatial characterization of\n  the point-source response) you would want to insert a single beam-sized \n  point source. Alternatively, you can insert one or more Gaussian-shaped \n  source(s) with a range of FWHMs to create a simulated source structure that\n  resembles the structure you expect your source(s) to have.\n\n  Make sure your test source is bright enough to see with high S/N, but not\n  too bright to trigger unintended flagging, or despiking. In general, a S/N\n  between 100 and 1000 should be fine for default reductions, and 100 to 300 \n  for `faint` or `deep` modes.\n\n  Additionally, in `faint` or `deep` modes, you may want to disable some \n  features which may affect your relatively bright test sources differently\n  than your much fainter real science target(s). Thus, the following are\n  recommended for reducing `faint` and `deep` test sources:\n\n\t\u003e crush [...] -blacklist=clip,blank,source.filter.blank\n\n  Note also that the spatial filtering (transfer function) will be varying\n  with location on the map (since the scanning speed and directions will\n  themselves be non-uniform over the map). Therefore, it is strongly \n  recommended that test sources are inserted near the same locations as the\n  real sources in the field.\n    \n\n\n\n### 2.5. Image processing post-reduction\n\n CRUSH also provides `imagetool` for post-processing images after reduction. \n The typical usage of imagetool is:\n\n\t\u003e imagetool -name=\u003coutput.fits\u003e [options] \u003cinput.fits\u003e\n\n which processes `\u003cinput.fits\u003e` with the given options, and writes the result \n to `\u003coutput.fits\u003e`. \n\n With imagetool, you can apply the desired level of smoothing (`-smooth`), or\n spatial filtering (`-extFilter`), specify circular regions to be skipped by \n the filter (`-mask=\u003cfilename\u003e`). You can adjust the clipping of noisy map \n edges by relative exposure (`-minExp`) or by relative noise (`-maxNoise`).\n You can also crop a rectangular region (`-crop=dx1,dy1,dx2,dy2`). There are \n many more image operations. See the imagetool manual (in your UNIX shell, or\n online) or simply run `imagetool` without an argument:\n\n\t\u003e imagetool\n\n One of the useful options allows to toggle between the noise estimate from \n the detector time-streams (`-noise=data`) and from the image itself \n (`-noise=image`) using a robust estimator. For example, after spatial \n filtering, you probably want to re-estimate the map noise:\n   \n\t\u003e imagetool [...] -extFilter=45.0 -noise=image \u003cinput.fits\u003e\n\n The built-in image display tool `show` also takes all processing options\n of imagetool, but rather than writing the result, it will display it in\n an interactive window. See the manual page of `show` (either inside your\n UNIX shell, or online), or run `show` without an argument.\n\n\n\n    \n### 2.6. Reducing Very Large Datasets\n\n Coming soon...\n\n\n---------------------------------------------------------------------------\n\n\n\n## 3. Advanced Configuration\n\n\nIn this section, you can find information on the most useful configuration\noptions. A complete list of available settings can be found in the\ntext file GLOSSARY, located in the CRUSH installation directory.\n\nConfiguration options here are listed as scripting keys i.e., without a \npreceding dash. However, you can use the same options in the command line by \nadding the dash. (Also, `=` can be replaced by space(s) in scripting...) \n\n\n### 3.1. Commands\n\n There are a handful of keywords that are treated as commands by crush. (The\n difference being that commands are interpreted and acted on right away, whereas \n typical configuration keys are stored settings that are interpreted later as \n necessary.) The commands are:\n   \n\n   \tconfig=\u003cfilename\u003e\tLoad the configuration file filename. \n\t\t\t\tThe file is looked for in the locations in the\n\t\t\t\tfollowing order:\n\n\t\t\t\t\t1. ./\n\t\t\t\t\t\n\t\t\t\t\t2. ./\u003cinstrument\u003e/\n\n\t\t\t\t\t3. ~/.crush2/\n\n\t\t\t\t\t4. ~/.crush2/\u003cinstrument\u003e/\n\n\t\t\t\tWhenever a matching file is found its contents\n\t\t\t\tare parsed. Because of the ordering, it is \n\t\t\t\tconvenient to create overriding configurations.\n\t\t\t\tThus instrument specific settings can be used \n\t\t\t\tto override default settings, and user specific\n\t\t\t\tsettings placed in '~/.crush2' can override\n\t\t\t\tshipped defaults. Whenever a configuration is\n\t\t\t\tparsed, there is a note of it on the console\n\t\t\t\toutput so that one always knows which files \n\t\t\t\twere read and in what order.\n\t\t\t\tE.g. when using\n\t\t\t\t\n\t\t\t\t   \u003e crush laboca -faint 12066\n\n\t\t\t\tthe following configuration files will be\n\t\t\t\tloaded in order (provided they exist):\n\n\t\t\t\t   \u003ccrush\u003e/config/default.cfg\n\t\t\t\t   \u003ccrush\u003e/config/laboca/default.cfg\n\t\t\t\t   ~/.crush2/default.cfg\n\t\t\t\t   ~/.crush2/laboca/default.cfg\n\t\t\t\t  \n\t\t\t\t   \u003ccrush\u003e/config/faint.cfg\n\t\t\t\t   \u003ccrush\u003e/config/laboca/faint.cfg\n\t\t\t\t   ~/.crush2/faint.cfg\n\t\t\t\t   ~/.crush2/laboca/faint.cfg\n\t\t\t\t   \n\t\t\t\tEach successively loaded file may override\n\t\t\t\tthe options set before it.\n\t\t\t\tWhen a matching configuration file is not found\n\t\t\t\tin any of the standard locations (above), CRUSH\n\t\t\t\twill make one last attempt to interpret the\n\t\t\t\targument as a regular pathname. This allows\n\t\t\t\tusers to store and invoke configuration files\n\t\t\t\tanywhere on the filesystem.\n\t\t\t\t\n\tforget=\u003coption\u003e...\tForget the priorly set values for \u003coption\u003e\n\t\t\t\tas if it were never defined. E.g. \n\t\t     \t\t\n\t\t\t\t\tforget=outpath\n\n\t\t\t\twill unset the 'outpath' option.\n\t\t\t\tYou can specify more than one options as a\n\t\t\t\tcomma-separated list. E.g.\n\t\n\t\t\t\t\tforget=outpath,project\n\t\t\t\t\n\t\t\t\tWith unset both the 'outpath' and 'project' \n\t\t\t\toptions.\n\t\t\t\tAdditionally, the arguments 'conditions' and\n\t\t\t\t'blacklist' can be used to clear the \n\t\t\t\tconditional or blacklisted settings \n\t\t\t\trespectively\t\n\n\trecall=\u003coption\u003e\t\tUndoes 'forget', and reinstates the \u003coption\u003e\n\t\t\t\tto its old value.\t\n\n\tremove=\u003coption\u003e\t\tSimilar to 'forget', but removes the entire\n\t\t\t\tbranch. Thus '-remove=despike' unsets:\n\n\t\t\t\t\tdespike\n\t\t\t\t\tdespike.level\n\t\t\t\t\tdespike.method\n\t\t\t\t\tdespike.flagfraction\n\t\t\t\t\t...\n\n\treplace=\u003coption\u003e\tUndoes the 'remove' option, reinstating the\n\t\t\t\t\u003coption\u003e tree to its prior state.\n\n\tblacklist=\u003coption\u003e...\tSimilar to 'forget', except it will not\n\t\t\t\tset options even if they are then specified\n\t\t\t\tat a later time. This is useful for altogether\n\t\t\t\tremoving settings from the configuration.\n\n\twhitelist=\u003coption\u003e...\tRemove \u003coption\u003e from the blacklist, allowing\n\t\t\t\tit to be set again if desired. Whitelisting\n\t\t\t\tan option will not reinstate it to its prior\n\t\t\t\tvalue. After whitelisting, you must explicitly\n\t\t\t\tset it again, or 'recall' or 'replace' it\n\t\t\t\tto its prior state.\n\n\tpoll\t\t\tWhenever unsure what options are set at any\n\tpoll=\u003coption\u003e\t\tgiven stage, you can poll the settings.\n\t\t\t\tWithout an additional argument it will list\n\t\t\t\tall options to the standard output. When\n\t\t\t\tan argument is specified it will list\n\t\t\t\tall configuration settings that start with\n\t\t\t\tthe specified string. E.g.\n\n\t\t\t\t\t\u003e crush [...] -poll=iter\n\t\n\t\t\t\twill list all iteration based options that \n\t\t\t\tare set including all the [...] options set\n\t\t\t\tprior to '-poll' in the command line.\n\n\n\tlist.divisions\t\tList all pixels divisions, which can be\n\t\t\t\tdecorrelated for the instrument.\n\n\tlist.response\t\tList all response modalities for the\n\t\t\t\tinstrument (to known signals, such as\n\t\t\t\ttelescope movement, or temperature data).\n\n\n### 3.2. Pipeline Configuration\n\n#### a. Source types. \n\n  The default reduction (see `default.cfg`) is optimized \n  for mapping compact (up to the field-of-view or smaller) sources in the S/N\n  range of \\~10-1000. These options are useful if your source does not match\n  these criteria.\n\n\n\tbright\t\tUse for bright sources (S/N \u003e ~1000). This setting\n\t\t\tentirely bypasses all filtering to produce a very\n\t\t\tfaithful map. The drawback is more noise, but\n\t\t\tthat should not be an issue for such a bright guy :-)\n\t\t\tWill invoke 'bright.cfg'.\n\n\tfaint\t\tUse with faint sources (S/N \u003c ~30) when the\n\t\t\tsource is faint but still visible in a single scan. \n\t\t\tThis setting applies some more aggressive filtering of \n\t\t\tthe timestreams, and extended structures. It invokes \n\t\t\t'faint.cfg'.\n\n\tdeep\t\tUse for very faint sources which are not at all \n\t\t\tdetected in single scans, or if you think\n\t\t\tthere is too much residual noise (baselines) in your \n\t\t\tmap to your liking. This setting results in the most \n\t\t\tagressive filtering. Will load the configuration from\n\t\t\t'deep.cfg'. The output map is optimally filtered\n\t\t\t(smoothed) for point sources.\n\n\textended\tTry to better preserve extended structures. This\n\t\t\tsetting can be used alone or in combination with\n\t\t\tthe above brightness options. See also '-sourcesize=X' \n\t\t\tbelow. With the fainter settings the recovery of \n\t\t\textended structures becomes increasingly more \n\t\t\tdifficult. For bright structures recovery up to FOV \n\t\t\t(or beyond!) should be possible, while for faint \n\t\t\tstructures \\~1/4 FOV - FOV scales are maximally \n\t\t\tobtainable (see more on this in the section below.)\n  \n\tsource.type=\u003ctype\u003e\tBy default, crush will try to make a map from\n\t\t\t\tthe data. However, some istruments may take\n\t\t\tdata that is analyzed differently. For example, you \n\t\t\tmay want to use crush to reduce pixel maps (to\n\t\t\tdetermine the positions of your pixels on sky), or\n\t\t\tskydips (to derive appropriate opacities), or do\n\t\t\tpoint source photometry. Presently, the following\n\t\t\tsource types are supported accross the board:\n\t\t\n\t\t\t   map \t\tMake a map of the source (default)\n\n\t\t\t   skydip\tReduced skydips, and determine \n\t\t\t\t\topacities by fitting a model to it.\n\n\t\t\t   pixelmap\tCreate individual maps for every\n\t\t\t\t\tpixel, and use it to determine their\n\t\t\t\t\tlocation in the field of view.\n\t\t\t\n\t\t\tNote, that you may also just use 'skydip' and \n\t\t\t'pixelmap' shorthands to the same effect. E.g.\n\n\t\t\t  \u003e crush [...] -skydip [...]\n\n\tsourcesize=X\tThis option can be used instead of 'extended' in \n\t\t\tconjunction with 'faint' or 'deep' to specify the \n\t\t\ttypical size of sources (FWHM in arcsec) that are \n\t\t\texpected. The reduction then allows filtering \n\t\t\tstructures that are much larger than the specified \n\t\t\tsource-size...\n\t\t\tIf 'sourcesize' or 'extended' is not specified, then \n\t\t\tpoint-like compact sources are assumed.\t\n\n\n#### b. Other commonly used pipeline settings:\n\n  (in typical order of importance to users...)\n\n\n\toutpath=\u003cpath\u003e\tSet the directory into which output files (e.g. maps) \n\t\t\twill be written. Can use '\\~' for home directory and \n\t\t\tenvironment variables in {}'s. Thus,\n\n\t\t\t   outpath=\\~/images\n\n\t\t\tand\n\n\t\t\t   outpath={$HOME}/images\n\n\t\t\tare equivalent\n\n\t\n\trounds=N\tIterate N times. You may want to increase the number\n\t\t\tof default iterations either to recover more extended\n\t\t\temission (e.g. when 'extended' is set), or to go\n\t\t\tdeeper (esp. when the 'faint' or 'deep' options are\n\t\t\tused).\n\n\n\titeration.[N]\tUse as a condition to delay settings until the Nth\n\t\t\titeration. E.g.\n\n\t\t\t   iteration.[3] smooth halfbeam\n\n\t\t\tor \n\n\t\t\t   \u003e crush [...] -iteration.[3]smooth=halfbeam [...]\n\t\t\t \n\t\t\tto specify half-beam smoothing starting from the 3rd\n\t\t\titeration.\n\t\t\t\n\titeration.[last]    Specify settings that should be interpreted only\n\t\t\t    at the beginning of the last iteration.\n\n\tfinal:key=value\t    A shorthand for the above :-).\n\t\n\titeration.[last-N]  Activate settings N iterations before the last\n\t\t\t    one.\n\n\titeration.[xx%]\t    Activate settings as a percentage of the total\n\t\t\t    number of iterations (as set by 'rounds'). E.g.\n\t\t\t\n\t\t\t\titeration.[50%] forget clip\n\n\t\t\t    can be used to disable the S/N clipping of the\n\t\t\t    source map half way through the reduction.\n\n\t\t\tBecause of the flexible syntax, the same iteration\n\t\t\tcan be referred to in different ways. Consider\n\t\t\ta reduction with 10 rounds. Then,\n\n\t\t\t\titeration.[5] smooth 5.0\n\t\t\t\titeration.[50%] smooth 10.0\n\t\t\t\titeration.[last-5] smooth beam\n\n\t\t\tcan all be used to define what happens in the 5th\n\t\t\titeration. CRUSH will parse these conditionals in \n\t\t\tthe above order: first the explicit iteration settings\n\t\t\tthen those relative to the reduction length, and\n\t\t\tfinally the settings relative to the end of the\n\t\t\treduction. Thus, in the above example the beam \n\t\t\tsmoothing will always override the other two settings.\n\n\tidle=N\t\tInstruct crush NOT to use N number of CPUs of the \n\t\t\tmachine. By default crush will try to use all \n\t\t\tprocessors in your machine for maximum performance. \n\t\t\tThis option allows to modify this behavior according \n\t\t\tto need. Note, that at least 1 CPU will always be used \n\t\t\tby crush, independent of this setting.\n\t\t\tThe number of actual parallel threads will be the \n\t\t\tsmaller of the allowed number of CPUs and the number \n\t\t\tof scans processed.\n\n\n\n\n### 3.3. Source (Map) Configuration\n\t\n\taltaz\t\tShorthand for 'system=horizontal' to reduce in Alt/Az.\n\t\t\tIt is also aliased to 'horizontal'.\n\n\tgrid=X\t\tset the map pixelization to X arcsec. Pixelization\n\t\t\tsmaller than 2/5 beam is recommended. The default is\n\t\t\t\\~1/5 beam pixelization.\n\n\tname=\t\tSpecify the output image file name, relative to the\n\t\t\tdirectory specified by 'outpath'. When not given\n\t\t\tminicrush will chose a file name based on the source\n\t\t\tname and scan number(s), which is either\n\n\t\t\t\t\u003csourcename\u003e.\u003cscanno\u003e.fits\n\n\t\t\tor\n\t\n\t\t\t\t\u003csourcename\u003e.\u003cfirstscan\u003e-\u003clastscan\u003e.fits\n\n\t\t\tFor mapping. Other source model types (e.g. skydips\n\t\t\tor pixel maps) may have different default naming \n\t\t\tconventions.\n\n\tpixelmap\tReduce pixel map data. Instead of making a single map\n\t\t\tfrom all pixels, separate maps are created for each\n\t\t\tpixel (Note, this can chew up some memory if you have\n\t\t\ta lot of pixels). At the end of the reduction CRUSH\n\t\t\tdetermines the actual pixel offsets in the focal plane.\n\t\t\tThis option is equivalent to 'source.type=pixelmap'.\n\t\n\tpixelmap.writemaps\tPixel maps (above) normally only produce the\n\t\t\t\tpixel position information. Use this option\n\t\t\tif you want CRUSH to write individual pixel maps as\n\t\t\twell, so you can peek at these yourself.\n\n\tprojection=\tChoose a map projection to use. The following \n\t\t\tprojections are supported:\n\n\t\t\t\tSFL  --  Sanson-Flamsteed\n\t\t\t\tSIN  --  Slant Orthographic\n\t\t\t\tTAN  --  Gnomonic\n\t\t\t\tZEA  --  Zenithal Equal Area\n\t\t\t\tSFL  --  Sanson-Flamsteed\n\t\t\t\tMER  --  Mercator\n\t\t\t\tCAR  --  Plate-Carree\n\t\t\t\tAIT  --  Hammer-Aitoff\n\t\t\t\tGLS  --  Global Sinusoidal\n\t\t\t\tSTG  --  Stereographic\n\t\t\t\tARC  --  Zenithal Equidistant\n\n\tskydip\t\tReduce skydip data, instead of trying to make an \n\t\t\timpossibly large map out of it :-). This option is\n\t\t\tequivalent to specifying 'source.type=skydip'.\n\n\tsmooth=X\tSmooth the map by X arcsec FWHM beam. Smoothing\n\t\t\thelps improve visual appearance, but is also useful\n\t\t\tduring reduction to create more redundancy in the data\n\t\t\tin the intermediate reduction steps. Also, smoothing\n\t\t\tby the beam is optimal for point source extaction from\n\t\t\tdeep fields. Therefore, beam smoothing is default in\n\t\t\twith the 'deep' option (see 'deep.cfg').\n\t\t\tTypically you want to use some smoothing during \n\t\t\treduction, and you may want to turn it off in the \n\t\t\tfinal map. Thus, you may have something like:\n\n\t\t\t  smooth=9.0\t\t\t# 9\" smoothing at first\n\t\t\t  iteration.[2]smooth=12.0 \t# smooth more later\n\t\t\t  iteration.[last]forget=smooth # no smoothing at last\n\n\t\t\tOther than specifying explicit values, you can use\n\t\t\tthe predefined values: 'minimal', 'halfbeam', '2/3beam'\n\t\t\tor 'beam'. See more on smoothing in the advanced\n\t\t\tconfiguration section.\n\n\tsource.filter   Filter extended structures. By default the filter will\n\t\t\tskip over map pixels that are above the 'blanking' S/N \n\t\t\tlevel (\u003e6 by default). Thus any structure above this \n\t\t\tsignificance level will remain unfiltered.\n\t\t\tFiltering is useful to get deeper in the map when \n\t\t\tretaining the very faint extended structures is not \n\t\t\tan issue. Thus filtering above 5 times the source size\n\t\t\t(see 'sourcesize') is default when the filter is used.\n\t\t\tSee the advanced configuration section for further\n\t\t\tdetails on fine tuning the large-scale structure \n\t\t\tfilter.\t\n\n\tsource.fixedgains\tSpecifies to use the fixed source gains \n\t\t\t        (e.g. from an RCP file -- see 'rcp' key).\n\t\t\tNormally, crush calculates source gains based on the \n\t\t\tcorrelated noise response and the specified point\n\t\t\tsource couplings (e.g. as derived from the two gain\n\t\t\tcolumns of RCP files.)\n\n\tsystem=\u003ctype\u003e\tSelect the coordinate system for mapping. The default\n\t\t\tis 'equatorial'. Other possibilities are 'horizontal'\n\t\t\t'ecliptic', 'galactic' or 'supergalactic'. Each of \n\t\t\tthese values is additionally aliased to simple keys. \n\t\t\tThus, you may use:\n\t\t\t\n\t\t\t   \u003e crush -galactic [...]\n\t\t\t   \n\t\t\tas a shorthand for '-system=galactic'.\n\t\t\t       \n\tunit=xxx\tSet the output to units xxx. You can use either the\n\t\t\tinstrumental units (e.g. 'V/beam' or 'counts/beam') or\n\t\t\tthe more typical 'Jy/beam' (default), as well as their\n\t\t\tcommon multiples (e.g. 'uJy/beam', or 'nV/beam').\t\n\n\n\n\n### 3.4. Scan Configuration\n\n Some options relate to the scans, helping to configure and handle them\n These options are specified *before* the list or range of scans to\n which they apply, and remain valid for all scans read after, or until\n an overriding option is placed. E.g.\n\n\t\u003e ./crush -option1=x 10218-10532 12425 -option2=y 11411 \\\n\t          -option1=z 10496\n\n will set option1 to 'x' for all scans but the last one, which will have\n this option set to 'z'. And the last two scans will have option2\n set to 'y'.\n\n A detailed listing of all scan specific options can be found in the \n 'GLOSSARY'. Here are a few of the most commonly used ones (in alphabetical\n order).\n\n\n\tpointing=dx,dy\t  Specify incremental pointing offsets x,y in the\n\t\t\t  system of the telescope mount (I.e., azimuth and\n\t\t\t  elevation for horizontal mounts. (Note, this option\n\t\t\t  works like pcorr in APECS). \n\n\tdatapath=\u003cdir\u003e\t  Start looking for raw data in directory \u003cdir\u003e. Some\n\t\t\t  instruments may also interpret it as a root directory\n\t\t\t  in which data may reside some specific hierarchy. E.g.\n\t\t\t  in \u003cdir\u003e/\u003cproject\u003e for APEX bolometers. Thus, if an\n\t\t\t  APEX instrument defines:\n\t\t\t\t\n\t\t\t\tdatapath /homes/data\n\t\t\t\tproject T-79.F-0002-2007\n\n\t\t\t  then crush will try to find data first in\n\t\t\t  '/homes/data', then in '/homes/data/T-79.F-0002-2007'\n\t\t\t  This provides a convenient way for accessing\n\t\t\t  hierarchically stored data. See the instrument-\n\t\t\t  specific usage of 'datapath' in the GLOSSARY.\n \n\tjackknife\t  Jackkniving is a useful technique to produce accurate\n\t\t\t  noise maps from large datasets. When the option is \n\t\t\t  used the scan signals are randomly inverted, s.t. the\n\t\t\t  source signals in large datasets will tend to cancel \n\t\t\t  out, leaving one with pure noise maps.\n\n\tproject=\u003cid\u003e      Some instruments (e.g. APEX bolometers) may require \n\t\t\t  a project ID to be set in order to locate scans by \n\t\t\t  serial number. Use capitalized form when defining \n\t\t\t  APEX projects. E.g.,\n\n                              project  T-79.F-0002-2007\n\n\tread \u003cfilename\u003e\t  Read the scan data from \u003cfilename\u003e, which can be \n\t\t\t  either a fully specified path, or relative to\n\t\t\t  'datapath'. (On the command line it is sufficient\n\t\t\t  to list the filename without a preceding dash.)\n\n\tread scanNo\t  Read scan number scanNo (scripting only).\n\t\t\t  in command line mode, ommit '-read='.\n\n\tread from-to\t  Read the range of scan numbers between from and to.\n\t     \t\t  (inclusive). On the command line, you can omit \n\t\t\t  '-read=' and simply list the scan range.\n\n\tread X Y [...]    You can combine scan numbers and different ranges\n\t\t\t  in a space-spearated list...\n\t\t\t\n\tread...\t\t  As mentioned, the 'read' keys only apply to scripts.\n\t\t\t  Thus, \n\n\t\t\t\tread 10755-10762             # in script\n\n\t\t\t  and\n\n\t\t\t\t\u003e ./crush [...] 10755-10762  # command line\n\n\t\t\t  are equvivalent.\n\t\t\t\t\n\tscale=X\t\t  Scale the fluxes by X. With this option you can apply\n\t\t\t  calibration corrections to your data.\n\n\tscale=\u003cfilename\u003e  Alternatively, scan specific scaling can be defined\n\t\t\t  by an appropriate calibration file, which among \n\t\t\t  other things, contains the ISO time-stamp and\t\n\t\t\t  the corresponding calibration values for each scan.\n\t\t\t  The filename may contain references to environment\n\t\t\t  variables enclosed in {} brackets. E.g.:\n\n\t\t\t    scale={$HOME]}/laboca/scaling.dat\n\n\tscramble\t  Another technique for generating noise maps, which \n\t\t\t  can be used also for small datasets, for which\n\t\t\t  jackknifing cannot be used. This approach scrambles\n\t\t\t  the pixel positions, such that the source signals\n\t\t\t  will be smeared out in the maps.\n\n\ttau=X\t\t  Specify a zenith tau value to use. When not used\n\t\t\t  minicrush will try to interpolate from\n\t\t\t  \u003cinstrument\u003e/\u003cinstrument\u003e-taus.master.dat if possible\n\t\t\t  or use 0.0 as default.\n\n\ttau=\u003cfilename\u003e\t  Alternatively, tau can also specify a file-name with \n\t\t\t  lookup information (usually containing tau\n\t\t\t  values from the radiometer or from the skydips).\n\t\t\t  Tau values will be interpolated for each scan,\n\t\t\t  as long as the scan falls inside the interpolator's\n\t\t\t  range. Otherwise, tau of 0.0 will be used. The \n\t\t\t  filename may contain references to environmnent \n\t\t\t  variables enclosed in {} brackets. E.g.:\n\n\t\t\t    tau={$HOME}/laboca/tau.dat\n\n\n\n### 3.5. Instrument Configuration\n\n There are various instrument configuration files. These reside in the\n corresponding instrument subdirectories inside the main crush directory.\n Some types of files are commonly used among many or all instruments, while \n others may be strongly instrument specific. \n\n In most cases the instrument configurations should be set correctly, and \n you probably can leave these settings alone. However, here you will find\n some of the most common instrument configuration options that you may, at \n times, want to adjust to your preference. \n\n A more complete list of the available instrument-specific configuration\n options can be found in the GLOSSARY.\n\n\n#### Generic Instrument Parameters\n\n\n\tpixeldata=\u003cfilename\u003e\tSpecifies a pixel data file, providing initial\n\t\t\t\tgains, weights and flags for the detectors,\n\t\t\tand possible other information as well depending on the\n\t\t\tspecific instrument. Such files can be produced via the\n\t\t\t'write.pixeldata' option (in addition to which you\n\t\t\tmay want to specify 'forget=pixeldata' s.t. flags are\n\t\t\tdetermined without prior bias).\n\n\trcp\t\tSpecify pixel positions (and optionally point-source\n\t\t\tand sky-noise gains). These are standard IRAM\n\t\t\tor APEX RCP files containing the information in ASCII\n\t\t\tcolumns. RCP files can be produced by the 'pixelmap' \n\t\t\toption, from scans, which move a bright point source \n\t\t\tover all pixels.\n\n\trcp.gains\tUse gains from the RCP files. Otherwise gains may \n\t\t\tcome from the 'pixeldata' file, or assume default\n\t\t\tvalues, such as 'uniform'.\n\n\trcp.center=x,y\tDefine the center RCP position at x,y in arcseconds.\n\t\t\tCentering takes place immediately after the parsing\n\t\t\tof RCP data.\n\n\trcp.rotate=X\tRotate the RCP positions by X degrees (counter \n\t\t\tclockwise). Rotations take place after centering (if\n\t\t\tspecified).\n\n\trcp.zoom=X\tZoom (rescale) the RCP position data by the scaling\n\t\t\tfactor X. Rescaling takes place after the centering\n\t\t\t(if defined).\t\n\n\tstability=X\tSpecify the instrument 1/f stability time-scale in\n\t\t\tseconds. This is used for determining optimal 1/f\n\t\t\tfiltering of the time-streams in the reduction.\n\n\n\n### 3.6. Advanced startup environment and Java configuration\n\n We have seen before how to configure the startup environment and Java by\n placing configuration snipplets inside the directory `~/.crush2/startup/`. \n However, what if you want certain settings to apply to just a specific \n program within the crush suite (e.g. to `crush` or `show`) only, but not \n to all of them? That too is possible, by (creating and) editing a file\n under the corresponding `~/.crush2/startup/\u003cprogname\u003e` directory. E.g.:\n\n\t~/.crush2/startup/crush/myconfig.conf\n\n will source the bash snippled contained in `myconfig.conf` for reductions\n (`crush` executable) only. As with other startup scripts, the name of the\n file itself is irrelevant, and all files will be parsed inside the \n program's startup directory (albeit in unspecified order!).\n\n Inside this file you can set up environment variables, and add shell \n (bash) scripts. E.g., the `crush` startup configuration file may contain \n the lines:\n\n\tCRUSH_NO_UPDATE_CHECK=\"1\"\n\tEXTRAOPTS=\"$EXTRAOPTS -Djava.awt.headless=true\"\n\n (The first disables update checking, the second adds headless mode to the\n list of extra Java options already defined).\n\n\n\n--------------------------------------------------------------------------\n  \n\n## 4. Correlated Signals\n\n\nThe removal of correlated signals, atmospheric or instrumental, is really the\nheart of CRUSH. In its most generic form the decorrelation is a two-step\nprocess: the first step finds and removes the correlated signals, assuming\nsome initial detector gains to it, after which the gains are estimated during \nthe second step based on the individual detector responses to the bolometer\nsignals. (For details, see the CRUSH paper: Kovacs 2008, Proc. SPIE, 7020, 45.)\nFlagging of non-responsive pixels, based on outlying gains (i.e. responses to\nthe correlated signals) may be part of the second step of the decorrelation.\n\n\n\n### 4.1. Modes and Modalities\n\n For each instrument, crush defines a set of _modalities_ (i.e. a set of\n correlated modes) on which decorrelation may be performed. Each mode in a\n _modality_ affects a _group_ of pixels. Thus, the collection of pixel groups\n with related modes constitutes a _division_. For example, SHARC-2 has 12 \n detector rows (each with 32 pixels). The dividing of pixels into 12 groups,\n each representing a detector row, is a pixel _division_. Pixels in a given \n row respond to the same correlated _mode_, and so there are 12 row-related \n modes, which are bunched together in a _modality_.\n\n Some of the modalities (and pixel divisions) are common to all (or most)\n instruments. These are:\n\n   `\u003cmodality name\u003e` |\tDescription\n   ----------------|-------------------------------------------------------\n   `all`\t   |\tAll channels, regardless of their state\n   `live`\t   |\tAll channels that aren't dead.\n   `detectors`\t   |\tAll detectors (e.g. excluding resistors etc.)\n   `obs-channels`  |\tAll observing detectors (e.g. that see sky).\n   `gradients`\t   |\tA focal plane gradients of 'observing' detectors.\n   `blinds`\t   |\tBlind detectors (if exist)\n   `telescope-x`   |\tTelescope position in the _x_ direction (e.g. AZ)\n   `telescope-y`   |\tTelescope poistion in the _y_ direction (e.g. EL)\n   `accel-\u003cdir\u003e`   |\tAcceleration in some direction (see below).\n   `chopper-\u003cdir\u003e` |\tChopper motion in some direction (see below).\n  \n\n\n Above, the motion-related modalities may have the following directionalities:\n\n\n\n   `\u003cdir\u003e`  |  Description\n   ---------|-------------------------------\n   `x`      |  _x_ direction.\t\n   `y`      |  _y_ direction.\n   `x^2`    |  square of the _x_ coordinate.\n   `y^2`    |  square of the _y_ coordinate.\n   `|x|`    |  _x_ magnitude.\n   `|y|`    |  _y_ magnitude.\n   `mag`    |  vector magnitude.\n   `norm`   |  square of vector magnitude.\n\n\n\n\n### 4.2. Removing Correlated Signals\n\n The decorrelation step is invoked by:\n\n\tcorrelated.\u003cmodality-name\u003e\n\n For example, a decorrelation of the signals induced by the chopper \n displacement in x (e.g. SHARC-2) is invoked by:\n\n\tcorrelated.chopper-x\n\n (Of course, the same keyword must also appear in the pipeline 'ordering', \n for the step to actually take place during the reduction process -- \n otherwise crush will not know when to correlated.the chopper signals.)\n\n You can fine-tune how CRUSH deals with each correlated mode. For example, \n the following lines:\n\n\tcorrelated.obs-channels.resolution 1.0\n\tcorrelated.obs-channels.gainrange 0.3--3.0\n\n Specifies that atmospheric signals, which appear in all observing channels, \n should be correlated. only every second (vs. the default every available \n frame), and that any channel that exhibits a response outside of 0.3--3.0 \n times the 'average' response of the array, should be flagged, and ignored \n in the reduction afterwards.\n\n\n\n### 4.3. User-defined Modalities\n\n In addition to the most common modalities listed above, each instruments \n defines its specific ones (e.g. `rows` for SHARC-2 from the example above). \n See the README files in the instrument sub-directories of crush for a full \n list of  predefined instrument specific modalities. You can also define your \n own pixel groups, divisions and modalities using the `group` and `division` \n keywords. Here is an example:\n\n\tgroup.my-group-1 10,13,20-25\n\tgroup.my-group-2 40-56,78,80-82\n\tdivision.my-division my-group-1 my-group-2\n\n The first two lines defines two pixel groups (`my-group-1` and `my-group-2`) \n from pixel indexes and ranges, while the last line creates a division \n (`my-division`) from these pixel groupings. CRUSH will also create a \n correlated modality with the same name as the division. So, given the \n definitions above you can correlated.on `my-division` by:\n\n\tcorrelated.my-division\n\n\n\n## 5. Custom Logging Support\n\nCRUSH (as of version 2.03 really) provides a poweful scan/reduction logging\ncapability via the `log` and `obslog` keys. \n\n`log` writes the log entries for each scan *after* the reduction, whereas \n`obslog` does not reduce data at all, only logs the scans immediately after \nreading them. While both logging functions work identically, some of the \nvalues are generated during reduction, and therefore may not be available to\n`obslog`.\n\n\n\n\n### 5.1. Log Files and Versioning\n\n You can specify the log files to use with the `log.file` and `obslog.file`\n keys (the default is to use `\u003cinstrument\u003e.log` and `\u003cinstrument\u003e.obs.log` in \n the `outpath` directory). Equivalently, you can also set the filename \n directly as an optional argument to `log` and `obslog`:\n\n\t\u003e crush [...] -log=myLog.log [...]\n\n You can specify the quantities, and the formatting in which they will appear,\n using the `log.format` and `obslog.format` keys. Below you will find more\n information on how to set the number formatting of quantities and a list\n of values available for the logging.\n   \n A log file always has a fixed format, the one which was used when creating \n it. Therefore, a conflict may arise if the same log file is specified for \n use with a different format. The policy for resolving such conflicts can be\n set via the `log.conflict` and `obslog.conflict` keys, which can have one\n of the following values:\n\n\toverwrite   Overwrites the previous log file, with a new one in the\n\t\t    newly specified format\n\n\tversion\t    Tries to find a sub-version of the same log file (with .1,\n\t\t    .2, .3 ... extension added) in the new format, or create\n\t\t    the next available sub-version.\n\n The default behaviour is to assume versioning, in order to preserve \n information in case of conflicts.\n\n\n\n\n### 5.2. Decimal Formatting of Values\n\n Many of the quantities you can log are floating point values, and you have\n the possibility of controlling how these will appear in your log files. \n Simply put one of the formatting directives in brackets after the value\n to specifts its format. \n\n E.g. the keys `RA` or `RAh` will write the right-ascention coordinate either\n as radians or as hours, with the default floating point formats. However,\n `RA(h:2)` will write the value in human-readable `hh:mm:ss.ss` format, \n whereas `RAh(f3)` will express it as `hh.hhh`.\n\n You can choose from the following formats to express various quantities.\n Be careful, because not all formats are appropriate to all types of data.\n (For example, you should not try to format angles expressed in degrees with\n the DMS formatting capabilities of the `a` angle formats. Use these only\n with angles expressed in radians!)\n\n\n\td0...d9\t\tInteger format with 0...9 digits. E.g. 'd3' will write\n\t\t\tPi (3.1415...) as 003.\n\n\te0...e9\t\tExponential format with 0...9 decimals. E.g. e4 will\n\t\t\twrite Pi as 3.1415E0.\n\n\tf0...f9\t\tFloating point format with 0...9 decimals. E.g. f3 will\n\t\t\twrite Pi as 3.141.\n\n\ts0...s9\t\tHuman readable format with 0...9 significant figures \n\t\t\t(floating point or exponential format,\n\t\t\twhichever is more compact).\n\n\ta:0...a:3\n\tas0...as3\n\tal0...al3\tAngle format with 0...3 decimals on the seconds. E.g.\n\t\t\t'a:1' produces angles in ddd:mm:ss.s format. Use only\n\t\t\twith values expressed as radians (not in degrees!).\n\t\t\tAs such, 'a:1' will format Pi as '180:00:00.0'. The\n\t\t\tdifference between the 'a:', 'as', and 'al' formats is\n\t\t\tthe separators used between degrees, minutes and seconds\n\t\t\t(colons, symbols, and letters respectively).\n\n\th:0...h:3\n\ths0...hs3\n\thl0...hl3\tHour-angle format (e.g. for RA coordinate) with 0...3\n\t\t\tdecimals on the seconds. E.g. 'h:2' formats angles in\n\t\t\t'hh:mm:ss.ss' format. Use only with values expressed\n\t\t\tas radians (not in degrees!). As such 'h:2' will format\n\t\t\tPi as '12:00:00.0'. The difference between the 'h:', \n\t\t\t'hs', and 'hl' formats is the separators used between \n\t\t\thours, minutes and seconds (colons, symbols, and \n\t\t\tletters respectively).\n\n\t\t\tE.g. Pi will be:\n\n\t\t\t\th:1\t12:00:00.0\n\t\t\t\ths1\t12h00'00.0\"\n\t\t\t\thl1\t12h00m00.0s\n\n\tt:0...t:3\n\tts0...ts3\n\ttf0...tf3\tTime format with 0...3 decimals on the seconds. E.g.\n\t\t\t't:1' formats time in 'hh:mm:ss.s' format. Use only\n\t\t\ton time values expressed in seconds! The difference \n\t\t\tbetween the 't:', 'ts', and 'tl' formats is the \n\t\t\tseparators used between hours, minutes and seconds \n\t\t\t(colons, symbols, and letters respectively).\n\n\n\t\n\n### 5.3. Loggable Quantities\n\n Currently, CRUSH offers the following quantities for logging. Directives\n starting with '?' will log the values of configuration keys. Other \n quantitites reflect the various internals of the scan or instrument state.\n More quantities will be added to this list in the future, especially \n values that are specific to certain instruments only. Keep an eye out for\n changes/addititions :-).\n\n\n    ?\u003ckeyword\u003e       The value of the configuration \u003ckeyword\u003e. If the\n                     configuration option is not defined '---' is written.\n                     If the keyword is set without a value, then '\u003ctrue\u003e'\n                     is written.\n\n    AZ               Azymuth coordinate (in radians). E.g. 'AZ(a:1)'\n                     produces ddd:mm:ss.s formatted output. See also 'AZd' \n                     and 'EL'.\n\n    AZd              Azymuth coordinate in degrees. E.g. 'AZd(f2)'\n                     produces output in ddd.dd format. See also 'AZ' and \n                     'ELd'\n\n    channels         Number of channels processed in the reduction. See\n                     also 'okchannels' and 'maxchannels'.\n\n    chopeff          Chopper efficiency.\n\n    chopfreq         Chopper frequency (in Hz).\n\n    chopthrow        Chopper throw (2x amplitude).\n\n    creator          Software that created the data file (as stored by\n                     the FITS CREATOR keyword).\n\n    date             Date (and time) of the observation. E.g. \n                     'date(yyyy-MM-dd)'. The format follows the rules for \n                     Java's SimpleDateFormat class.\n\t\t\t\n    DEC              Declination coordinate (in radians). E.g. 'DEC(a:0)'\n                     produces the declination in ddd:mm:ss format. See also\n                     'RA' and 'DECd'.\n\n    DECd             Declination coordinate (in degrees). E.g. 'DECd(f1)'\n                     produces output in ddd.d format. See also 'RAh' and\n                     'DEC'.\n\n    descriptor       A descriptor string for the scan (e.g. the scan number\n                     or file name used to invoke the scan).\n\n    dir              Scanning direction. E.g. 'HO', 'EQ', 'EC', 'GL', 'SG'.\n\n    EL               Elevation coordinate (in radians). \n\t\n    ELd              Elevation coordinate (in degrees).\n\n    epoch            Full coordinate epoch, e.g. (J2000.0).\n\n    epochY           E.g. epochY(f1) -\u003e 2000.0\n\n    frames           Number of unflagged frames in the scan.\n\n    FWHM             Mean beam FWHM of the instrument (in 'sizeunit').\n\n    gain             Instrument gain.\n\n    generation       Source model generation.\n\n    hipass           Highpass filtering timescale (in seconds)\n\n    humidity         Ambient humidity (if recorded).\n\n    id               Scan identifier (e.g. scan number)\n\n    integrations     Number of integrations (subscans) contained in the \n                     scan.\n\n    LST              Local Sidereal Time (in seconds). E.g. use 'LST(t:1)'\n                     to format is as 'hh:mm:ss.s'.\n\n    LSTh             LST in hours.\n\n    maxchannels      Maximum number of channels the instrument can have\n                     (effectively the number of channels stored in the\n                     data file).\n\n    maxFWHM          Smallest beam of the instrument (in sizeunit).\n\n    minFWHM          Largest beam of the instrument (in sizeunit).\n\n    MJD              Modified Julian Date of the scan.\n\n    mount            The focus in which the instrument is mounted.\n\n    NEFD             Average Noise Equivalent Flux Density (Jy sqrt[s]) as a\n                     measure of the array sensitivity.\n\n    object           Name of observed object.\n\n    observer         Name(s) of the observer(s).\n\n    obshours         Effective on-source time (in hours).\n\n    obsmins          Effective on-source time (in minutes).\n\n    obstime          Effective on-source time (in seconds).\n\n    okchannels       Number of good (unflagged) channels.\n\n    PA               Mean parallactic angle (in radians).\n\n    PAd              Mean parallactic angle in degrees.\n\n    pnt.angle        Source elongation angle on map (degrees).\n\n    pnt.asymX        Asymmetry in telescope X (%). \n\n    pnt.asymY        Asymmetry in telescope Y (%). `\n\n    pnt.AZ           Azymuth pointing (arcsec).\n\n    pnt.EL           Elevation pointing (arcsec).\n\n    pnt.DEC          Declination pointing (arcsec).\n\n    pnt.dasymX       Asymmetry uncertainty in telescope X (%).\n\n    pnt.dasymY       Asymmetry uncertainty in telescope Y (%). `\n\n    pnt.dangle       Source elongation angle error (deg).\n\n    pnt.dAZ          Azymuth pointing residual (if available).\n\n    pnt.dDEC         Pointing residual in the declination dir (if available).\n\t\n    pnt.dEL          Elevation pointing residual (if available).\n\n    pnt.delong       Source elongation error (%).\n\n    pnt.delongX      Source elongation error in telescope X direction (%).\n\n    pnt.dNasX        Pointing residual in the Nasmyth X dir (if available).\n\n    pnt.dNasY        Pointing residual in the Nasmyth Y dir (if available).\n\n    pnt.dRA          Pointing residual in the RA direction (if available).\n\n    pnt.dX           Pointing residual in native X direction.\n\n    pnt.dY           Pointing residual in native Y direction.\n\n    pnt.elong        Source elongation (%).\n\n    pnt.elongX       Source elongation in telescope X direction (%).\n\n    pnt.RA           RA pointing (arcsec).\n\n    pnt.X            Aggregated X pointing correction (including applied\n                     corrections and sometimes the pointing model too).\n\n    pnt.Y            Aggregated Y pointing correction (including applied\n                     corrections and sometimes the pointing model too).\n\n    pressure         Ambient pressure (if recorded) in hPa.\n\n    project          Project name (if defined)\n\n    ptfilter         The point-source flux filtering throughput of the \n                     reduction.\n\n    RA               Right-ascention coordinate (in radians). E.g. use\n                     'RA(h:2)' to format as 'hh:mm:ss.ss'.\n\n    RAd              Right-ascention coordinate (in degrees).\n\n    RAh              Righ-ascention coordinate (in hours).\n\n    rate             Sampling rate of the data (Hz). See also 'sampling'\n\n    resolution       Instrument resolution (i.e. FWHM of the main beam).\n                     in 'sizeunit'.\n\t\n    rmsspeed         RMS fluctuation of the scanning speed (arcsec/s).\n\n    sampling         Sampling interval (seconds). The inverse of 'rate'.\n\n    scale            Scaling factor applied to the scan. \n\n    scanspeed        Average scanning speed (arcsec/s).\n\n    serial           Serial number of the scan.\n\n    sizeunit         Size unit for measuring resolutions. E.g. 'arcsec'. \n\n    src.a            Major axis (arcsec) of the source ellipse.\n\n    src.angle        Orientation of the source ellipse (deg).\n\t\n    src.b            Minor axis (arcsec) of the source ellipse.\n\n    src.da           Uncertainty of the minor axis (arcsec).\n\t\n    src.dangle       Uncertainty of the orientation (deg).\n\n    src.db           Uncertainty of the major axis (deg).\n\n    src.dFWHM        Uncertainty of the source FWHM (arcsec).\n\n    src.dint         Uncertainty of the integrated source flux (Jy). \n\n    src.dpeak        Uncertainty of the peah source flux (Jy/beam).\n\n    src.FWHM         Source FWHM (arcsec).\n\n    src.int          Integrated source flux (Jy) insize and adaptive\n                     aperture.\n\n    src.peak         Peak source flux (Jy/beam).\n\n    stat1f           A measure of the 1/f noise averaged over the array.\n                     The configurations options '1overf.freq' and \n                     '1overf.ref' define the frequencies for the 1/f\n                     measurement and white-noise reference respectively.\n\n    Tamb             Ambient temperature (if recorded) in degrees C.\n\n    tau              The in-band, line-of-sight opacity value.\n\n    tau.\u003cid\u003e         The zenith tau value for \u003cid\u003e. E.g. 'tau.225GHz' or\n                     'tau.PWV' if these are defined by appropriate scaling\n                     relations.\n\n    UT               UT in seconds. E.g. use 'UT(t:1)' to format is as \n                    'hh:mm:ss.s'.\n\n    UTh             UT in hours.\n\n    weight          The relative weight of the scan, based on the actual\n                    noise of the map it generates.\n\n    winddir         Wind direction (if recorded) in degrees.\n\n    windpk          Peak wind speed (if recorded) in m/s.\n\t\n    windspeed       Wind speed (if recorded) im m/s.\n\n    zenithtau       In-band opacity at zenith.\n\n\n#### Source model specific log entries\n\n    map.beams        Number of (smoothed) beams in the map.\n\n    map.contentType  Type of data stored in the map.\n\n    map.creator      Creator's name or description.\n\t\n    map.depth        Weighted average depth of the map in map units.\n\n    map.max          Maximum value in map units.\n\n    map.min          Minimum value in map units.\n\n    map.name         Name of map data (e.g. object's name)\n\n    map.points       Number of pixels in the map.\n\n    map.rms          Typical RMS of the map in map units.\n\n    map.size         Size of the map in pixels e.g. '121x432'.\n\n    map.sizeX        Map size in the 'x' direction (pixels).\n\n    map.sizeY        Map size in the 'y' direction (pixels).\n\n    map.system       Coordinate system id, e.g. 'HO', 'EQ', 'GL' etc.\n\n    map.unit         Name of map unit, e.g. 'Jy/beam'.\n\n    phot.flux        Photometric flux in Jy/beam (e.g. for LABOCA).\n\n    phot.dflux       Uncertainty in the photometric flux (Jy/beam).\n\n    skydip.kelvin    Data units that corresponf to a 1K load on the detectors.\n                     (for skydip data, if and when fitted.)\n\n    skydip.dkelvin   Uncertainty in data units per kelvin conversion.\n\n    skydip.tau       Tau value derived (or assumed) from skydip data\n   \n    skydip.dtau      Tau uncertainty from skydip data\n\n    skydip.tsky      Derived (or assumed) sky temperature in K.\n\n    skydip.dtsky     Sky temperature uncertanity (K).\n\n    smooth           Smoothing applied, in the default size unit of\n                     the instrument (e.g. in arcsecs).\n\n\n\n\n#### Telescope specific log entries\n\n   APEX: see the `crush/Documentation/README.apex.\n\n\n#### Instrument specific log entries\n\n   See the `crush/Documentation/README.\u003cinstrument\u003e`, for each `\u003cinstrument\u003e`.\n  \n\t\n---------------------------------------------------------------------------\n\n\n## 6. Supported Cameras\n\nCurrently, CRUSH supports the following instruments (in alphabetical order):\n\n   \n    GISMO        (2mm) Goddard-IRAM Superconducting 2-Millimeter Observer\n                 www.iram.es/IRAMES/mainWiki/GoddardIramSuperconductingTwo\\\n                 Millimeter\n\n    LABOCA       (870um) Large APEX Bolometer Camera\n                 www.apex-telescope.org/bolometer/laboca\n\n    PolKa        (polarimetry) Polarimeter for LABOCA\n                 www.mpifr-bonn.mpg.de/staff/gsiringo/laboca/laboca\\\n                 _polarimeter.html\n\n    SABOCA       (350um) Submillimeter APEX Bolometer Camera\n                 www.apex-telescope.org/bolometer/saboca\n\n    SCUBA-2      (450um, 850um) Submillimetre Common User Bolometer Array 2\n                 www.roe.ac.uk/ukatc/projects/scubatwo\n\n    SHARC-2      (350um, 450um, 850um) The second-generation Submillimeter \n                 High-Angular Resolution Camera\n                 Caltech, Pasadena, CA\n                 www.submm.caltech.edu:/~sharc\n\n    SOFIA/HAWC+  (53um, 62um, 89um, 155um, 216um)\n                 High-resolution Airborne Wide-angle Camera\n\n\nThe following instruments have legacy support only (supported by CRUSH 2.4x\nand earlier releases):\n\n\n    ASZCa        (2mm) APEX SZ Camera, from Berkeley, CA\n                 bolo.berkeley.edu/apexsz/instrument.html\n\n    MAKO         (350um) KID technology demonstration camera for the CSO.\n\n    MAKO-2       (350um, 850um) Second-generation KID technology demonstration\n                 camera for the CSO with dual-pol pixel response, and dual-band \n                 imaging capability.\n\n    MUSTANG-2    (3mm) Large focal plane array for the 100m Greenbank Telescope.\n                 www.gb.nrao.edu/mustang/\n\n    p-ArTeMiS    (200um, 350um, 450um) 3-color camera for APEX (prototype)\n                 www.apex-telescope.org/instruments/pi/artemis\n\n    SHARC        (350um, 450um) Submillimeter High-Angular Resolution Camera\n                 Caltech, Pasadena, CA\n                 http://www.submm.caltech.edu/cso/sharc/cso_sharc.html\n\n\n\n\nFurther support for instruments is possible. If interested to use CRUSH for\nyour bolometer array, please contact Attila Kovacs \n`\u003cattila[AT]sigmyne.com\u003e`.\n\nIn principle, it is possible to extend CRUSH support for other types of scanning\ninstruments, like heterodyne receiver arrays, or single-pixel receivers that\nscan in frequency space, or for the reduction of line-surveys from double\nside-band (DSB) receivers. Such new features may appear in future releases, \nespecially upon specific request and/or arrangement...\n\n\n---------------------------------------------------------------------------\n\n\n## 7. Future Developments (A Wish List...)\n\n\nThere are a number of plans for new features in CRUSH. Some may come in the\nnear future, others perhaps later, depending on the resources available for\nimplementation. Nonetheless, the following avenues of feature enrichment are\nbeing considered. \n\n\n### 7.1. Support for Heterodyne Receivers and Interferometry\n\n CRUSH was born as a bolometer array reduction package. However, there is no\n reason why many of its principles could not be applied to other types of \n instruments (astronomical or otherwise), especially those, which are affected\n by correlated signals. Besides, CRUSH also provides powerful tools for other\n analysis steps, like weighting, despiking, spectral filtering, and mapping.\n\n Two obvious extensions would be heterodyne receivers (and receiver arrays) \n and the use of CRUSH for interferometry (e.g. ALMA) since it is well-suited\n to deal with immense data volumes also.\n\n\n### 7.2. Interactive Reductions\n\n At present CRUSH offers only a reduction pipeline. This is ideal for\n crunching large volumes of data in a more-or-less automated fashion, and \n for making reduction painless. It is also ideal for most users, who might \n not wish to learn about the intricacies of each and every reduction step. \n However, in some cases having more control may be beneficial.\n\n Many astronomers are used to interactive reduction packages (e.g. GILDAS\n AIPS, BoA etc.). CRUSH should eventually offer such a capability also. This\n would allow step-by-step reductions, together with varous plotting \n capabilities to look at data at every stage. Such a capability would be\n very useful in the process of building pipelines for new instruments\n\n It should be relatively simple to provide this feature. The current \n configuration languange of CRUSH can be easily adapted for more interactive\n use. Even more detailed control may be possible though a standard scripting\n language like JavaScript or Rhino. The main job would be to supply the\n essential plotting capabilities, but that too may come sooner than later...\n\n\n### 7.3. Distributed and GPU Computing\n\n The reduction paradigm of CRUSH is massively parallelizable. CRUSH can\n already make good use of any number of processing cores within a computer.\n It should be quite staighforward to extend implementation over several\n computing nodes and super-computers, allowing orders of magnitude increases\n in data reduction speeds and data volumes.\n\n Another way of improving speeds may come from the use of Graphical (GPU)\n Computing. Today's grahics chips provide computing power well in excess of\n that offered by the CPUs. This can be harnessed with programming tools\n like CUDA or OpenCL. GPU computing is still in its infancy, and thus\n its languange specifications are likely very fluid. But technical\n details aside, GPUs clearly offer a way for boosting the number crunching\n performace of CRUSH. \n\n\n### 7.4. Graphical User-Interface (GUI)\n\n The addition of a Graphical User Interface (GUI) to CRUSH would make \n configurations more transparent and intuitive. It would allow users, who do \n not use CRUSH often, to avoid learning the complexities of command-line\n options and scripting support, and instead click their way through the\n essential configuration settings.\n\n GUIs may also aid the more expert users, in providing a way to look at \n details, such as monitoring quantities, signals or maps during the reduction.\n\n\n-----------------------------------------------------------------------------\n\n## 8. Further information\n\n\nCRUSH-2 is the next-generation data reduction package, inheriting its DNA from\nthe pioneering SHARC-2 specific version (crush-1.xx) and from the APEX specific\nminicrush implementation. It is a much more capable package than either of its\npredecessors. However, because the output FITS images are backward compatible \nwith the  older versions (for the most part), parts of the original CRUSH \npackage can still be useful for manipulating images post reduction.\n\nMainly, the crush-1.xx distribution provides tools for displaying (`show`), \nminupulating (`imagetool`), and coadding (`coadd`) or differencing \nimages (`difftool`), and  producing histogram (`histogram`) from them.\n\nDownloading the CRUSH package(s), and further information on the FITS images\nis available at:\n\n  http://www.sigmyne.com/crush\n\nI hope you'll find this helpful and may you end up with beautiful images!!!\n\n\n-----------------------------------------------------------------------------\n\nCopyright (C)2019 -- Attila Kovacs \n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fattipaci%2Fcrush","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fattipaci%2Fcrush","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fattipaci%2Fcrush/lists"}