********************************************************************************
NUSTAR
********************************************************************************
NuSTARDAS version 1.2.0 - new software for the NuSTAR mission:
* nuattcorr - Correct NuSTAR attitude file
* nubackscale - Apply backscale correction to NuSTAR energy spectra
* nucalcpha - Correct energy, calculate PHA and assign event grades for NuSTAR event files.
* nucalcpi - Calculate PI column values of NuSTAR event files.
* nucalcpos - Convert Raw coordinates into Detector coordinates of NuSTAR event files.
* nucalcsaa - Calculate SAA passages for NuSTAR
* nucoord - Convert Raw coordinates into Detector and Sky coordinates of NuSTAR event files
* nuexpomap - Generate exposure maps for NuSTAR event
* nufilter - Create a NuSTAR filter file from HK data running 'prefilter' and 'makefilter'
* nuflagbad - Flag events for bad pixels in NuSTAR event files
* nuflagevt - Flag events according to various cuts criteria in NuSTAR event files
* nuhotpix - Search for hot and flickering pixels in NuSTAR event files
* nulccorr - Apply livetime correction to NuSTAR light-curves
* nulivetime - Apply the dead time correction to the temporal keywords of the NuSTAR event files.
* numetrology - Process the metrology data for NuSTAR
* numkarf - Generates an ARF file for an input PHA file for NuSTAR
* numkrmf - Generates an RMF file for an input spatial region file for NuSTAR
* nupipeline - Run in sequence the tasks for NuSTAR data processing
* nuproducts - Extract high-level data products from a Level 2 event file for NuSTAR
* nuscreen - Generate GTIs and use them together with event properties selection to screen NuSTAR event files
* nuskypos - Calculates SKY position of the optical axis, of the aperture stop center and of the DET1 reference pixel for NuSTAR
* nuskytodet - Calculates DET1 and DET2 coordinates of a SKY pixel for NuSTAR
* nuversion - Report version string for NuSTAR package
--------------------------------------------------------------------------------------------------------------------------------------
********************************************************************************
NUSTAR
********************************************************************************
NuSTARDAS version 1.3.0:
Major changes introduced in this version:
- numetrology: Addition of the new input parameter psdcal, with default value yes, to optionally (if set to no) disable the linearization of the PSD X/Y coordinates.
- nuscreen: for SCIENCE observing mode
1) introduction of a new run of the FTOOL maketime to calculate the ONTIME value during which the PSD coordinates are outside the CALDB metgridfile calibrated grid and
2) addition of a new keyword named NUPSDOUT to the output event file storing the ONTIME calculated value. See the NuSTARDAS users guide sections 4.5 and 6.3.
- numkarf:
0) for extended sources (parameter extended=yes) the input parameters grflag and psfflag are automatically set to no;
1) introduction of an energy dependent PSF correction;
2) fixed bug for small values of the input parameter boxsize (extended source case);
3) changed the call to the nuexpomap module.
- nuexpomap: introduction of a new algorithm for inclusion of bad/hot pixels in exposure maps.
- nulccorr:
1) correction for PSF losses using a CALDB PSF file corresponding to the value of the energy input parameter;
2) changed the call to the nuexpomap module.
- nubackscale: changed the call to the nuexpomap module;
- nupipeline:
1) addition of the new input parameter psdcal;
2) modified the call to the module numetrology;
3) addition of the ONTIME value during which the PSD coordinates are outside the CALDB metgridfile in the STDOUT final report; modified call to modules nuexpomap and nuproducts to accomodate above changes.
- nuproducts: modified the call to the modules nulccorr, numkarf and nubackscale.
Summary of changes:
numetrology:
Allows the pipeline to be run with the calibration of the PSDs off. This prevents data from time periods when the PSDs are out-of-range from being lost in the cleaned event data (for some observations this may be result in as much as a 10-20% increase in exposure). The effect on the event reconstruction varies source-by-source, but the reconstruction will in general, be worse with the calibration turned off. Turning off the PSD calibration may also affect the absolute position reconstruction.
nuscreen:
In the default setting the pipeline now appends a keyword to the cleaned event files, "NUPSDOUT", that contains the number of seconds lost to the PSD out-of-range filter. If this is a large fraction of the total observation, then the user may want to turn off the PDS calibration and re-run the pipeline. See the NuSTARDAS users guide section 6.3.
numkarf:
(0) Fixed a bug where the ghost-ray and PSF corrections defaulted to "ON" even for extended sources. These flags correctly default to "OFF" when extended=YES.
(1) Includes the effect of the energy-dependent PSF when psfflag=YES. This effect is on the ~5% and primarily acts below 10 keV for sources extracted using small (<<30 arcsecond radius) extraction regions.
(2) Bug fix.
(3) Keyword maintenance.
nuexpomap:
Improved the algorithm for recomputing exposure maps. Pixels that are disabled (either in software on by the user via the userbadpix file) are now properly vetoed in the exposure map.
nulccorr:
Applies energy-dependent PSF corrections as per numkarf.
--------------------------------------------------------------------------------------------------------------------------------------
********************************************************************************
NUSTAR
********************************************************************************
NuSTARDAS version 1.3.1:
Major changes introduced in this version:
- nuexpomap: addition of the keyword 'PIXBIN' and 'PERCENT' to the header of the output sky exposure maps.
- nulccorr: update of the value of the 'DEADAPP' keyword to 'True' in the output light-curve
- nupipeline: 1) addition of the new input parameters 'barycorr', 'clockfile', 'srcra_barycorr' and 'srcdec_barycorr'; 2) modified the call to the module 'nuproducts' to enable the barycenter correction
- nuproducts: 1) addition of the new input parameter 'barycorr' to optionally apply the barycenter correction to output files with internal runs of the multi-mission FTOOL 'barycorr'; 2) addition of the energy filtering of the output sky images through the input parameters 'pilow' and 'pihigh'.
Summary of changes:
* nupipeline/nuproducts:
Allows the pipeline to produce barycenter corrected event files and lightcurves via a new keyword passed to the "nuproducts" script. The user may set the RA/DEC of the object using the 'srcra_barycorr' and 'srcdec_barycorr' keywords to nupipeline and/or nuproducts.
Output sky images are now filtered using the "pilow" and "pihigh" keywords. Previously only the lightcurve was filtered and the images produced by nupipeline/nuproducts were always "full band" images and the user was required to use to XSELECT or "extractor" to produced energy-band images. This has been changed for simplicity.
* nulccorr:
Some lightcurve analysis suites (e.g. lcurve) read the "DEADAPP" keyword to determine whether or not the deadtime correction has been applied. Previously, the "DEADAPP" keyword had not been set, which could result in livetime corrections being applied twice (once in the bin-by-bin by nulccorr and then the average livetime correction across the lightcurve by lcurve or other XRONOS FTOOLs). DEADAPP is now correctly set if nulccorr has been called on a lightcurve and livetime corrections have been applied.
* nuexpomap:
The "PIXBIN and "PERCENT" option keywords have been added to the header of the exposure maps to improve self-documentation.
--------------------------------------------------------------------------------------------------------------------------------------
********************************************************************************
NUSTAR
********************************************************************************
NuSTARDAS v1.4.0:
- Major changes introduced in this version:
* nuexpomap: 1) addition of a new parameter named 'indet2instrfile' to allow the generation of sky exposure map starting from a DET2 instrument map; 2) algorithm optimization for execution time reduction;
* nubackscale: addition of a new parameter named 'inexpomapfile' to provide in input a sky exposure map. This change allows to skip the internal generation of exposure maps and reduces the task execution time;
* nulccorr: addition of two new parameters named 'inskyinstrfile' and 'inaspecthistofile' to provide in input a sky instrument map and the associated aspect histogram file. This change allows to skip the internal generation of sky instrument maps and reduces the task execution time;
* numkarf: addition of a new parameter named 'inexpomapfile' to provide in input a sky exposure map in the case of extended sources (parameter 'extended=yes'). This change allows to skip the internal generation of exposure maps and reduces the task execution time;
* nuproducts: 1) addition of the new input parameter 'rungrppha' to group the source energy spectrum through an internal run of the FTOOL 'grppha'; 2) addition of an internal run of the 'nuexpomap' module and modification of the calls to the 'nubackscale', 'nulccorr' and 'numkarf' modules;
* nucalcpi: modification of the algorithm for GRADE 0 and GRADE 9-20 events;
* nupipeline: 1) handling of compressed TLE files in input; 2) syntax modification to support latest Perl versions.
Summary of changes:
- nuexpomap/nubackscale/nulccorr/numkarf/nuproducts:
Performance improvement to avoid multiple runs of nuexpomap. The software now allows the user to store a single DET2 instrument map file (through the det2instrfile output keyword call to nuexpomap) and reuse this file for future runs, including changes to the vignetting energy, etc. See section 4.9 of the NuSTARDAS User's Guide for more details and syntax.
This has resulted in a dramatic speed increase for nuproducts runs, with decreases in run times of 20 to 80%, depending on the exact parameters.
- nucalcpi:
Slight change in the reconstruction algorithm for how events with Grade 0 and Grades 9 through 20 are handled in the software. This change allows for fine tuning of the GRADE_GAIN and GRADE_OFFSET columns in the CLC CALDB file. There are associated updates to the CLC CALDB files (see CALDB release notes) which marginally improve the gain correction for all events.
--------------------------------------------------------------------------------------------------------------------------------------
********************************************************************************
NUSTAR
********************************************************************************
- NuSTARDAS v1.4.1:
Major changes introduced in this version:
* nuproducts:
1) Modification of the internal run of the FTOOL 'grppha' to define a PI energy range by using two new input parameters 'grppibadlow' (replacing 'grppibad') and 'grppibadhigh';
2) Addition of the new input parameter 'usrgtibarycorr' to regulate the barycenter correction of the optional input user GTI file ('usrgtifile').
* nupipeline:
1) Modification of the call to the 'nuproducts' module;
2) Addition of the new input parameter 'productsgtibarycorr' to regulate the barycenter correction of the optional input user GTI file ('productsgtifile') for spectrum and light curve extraction.
Summary of changes:
- nuproducts (grppha):
Improvements to the user interface for nuproducts allowing the user to automatically apply grppha corrections to the output spectrum. Default values for the low/high bounds of "bad" channels in the grppha call are at 3 and 79 keV, which allows users to "ignore bad" in XSPEC to ignore data outside of the usable science range for NuSTAR. The user accepts responsibility for the effect of binning the data; we encourage the users to explore the literature on binning and the impacts of various fit parameters before using grppha.
- nuproducts (barycorr):
Improvement to the user interface for nuproducts allowing the user to specific a usrgti file in barycentric time (e.g. for individual pulses or bursts in a pulsar). By default, the pipeline assumes that usrgtis are given in NuSTAR-epoch time and are barycenter corrected.
--------------------------------------------------------------------------------------------------------------------------------------
********************************************************************************
NUSTAR
********************************************************************************
NuSTARDAS v1.5.1:
Major changes introduced in this version:
- numkarf: 1) addition of the new input parameters 'pilowarf' and
'pihigharf' to define the energy range used to calculate the box
ARF weights for the 'addarf' internal run (extended sources case);
2) addition of the new input parameter 'flatflag' to apply a flat
spatial counts distribution to calculate the box ARF weights for
the 'addarf' internal run (extended sources case).
- numkrmf: bug fix for addrmf execution when CALDB in remote mode
is used.
- nuproducts: addition of the new input parameters 'pilowarf',
'pihigharf' and 'flatflagarf' to support the numkarf changes.
- nupipeline: 1) addition of the new input parameters 'pilowarf',
'pihigharf' and 'flatflagarf' to support the numkarf changes;
2) handling of the cases 'createexpomap=yes' and/or 'exitstage=3'
when the '01' cleaned event file is empty.
Summary of changes:
- numkarf:
When observing extended sources it's often the case that the
high-energy photons have a different spatial distribution than
the low-energy photons. Prior to this patch, the ARF generator
used the spatial distribution for all photons over the allowed
NuSTAR energy range (1.6-165 keV) to determine the box ARF
weights. The new keywords (pilowarf and pighigharf) can be used
to set a specific energy range (e.g above >40 keV or 5-7 keV).
Recall that the conversion between energy and PI is :
PI = (keV - 1.6) / 0.04 channels.
Additionally, we have added the option to use a "flat" spatial
distribution over the source region (e.g. if the intrinsic source
distribution is unknown) which equally weights all boxes in the
source region when generating the ARF.
- nupipeline / nuproducts :
Added feed through keywords to support the above changes to numkarf.
Also improved handling of cases when high level (Stage 3)
science products are being produced but the SCIENCE data mode
file is empty. This commonly occurs during SLEW sequence IDs and
previously the pipeline would exit with an error.
--------------------------------------------------------------------------------------------------------------------------------------
********************************************************************************
NUSTAR
********************************************************************************
- Updates for NuSTARDAS 1.6.0:
Please note that this version requires NuSTAR CALDB dated 2016
03/15 or newer and will not work with older versions of the CALDB.
Major changes introduced in this version:
nusplitsc (NEW TASK):
When the primary star tracker (also known as a camera head unit, or
CHU) is blinded by the Earth limb or glint from the Sun the data is
recorded in 'Mode 06' by the pipeline. In this mode the pipeline
uses the spacecraft start rackers (CHUs 1, 2, and 3) to determine
the sky position of each count. This sky position can be subtly
different when different combinations of star trackers are guiding
the observatory pointing, resulting in blurring of the output image.
nusplitsc allows the user to generate separate independent output
files for each CHU combination, each of which should have reduced
blurring and higher fidelity. We note that the absolute position of
these images may still be different than the Mode 01 data set and do
not recommend using RA/DEC coordinates when extracting produces in
these modes without first inspecting the images.
See the Observatory Guide as well as Walton et al. (2016) and
Grefenstette et al. (2016) for more details.
- nucalcsaa: The SOC has optimized new algorithms to automatically
generate GTIs that exclude regions of the orbit that produce
enhanced background counts in the detector. See the Observatory
Guide for a full description of these new modes and their effects.
- nupipeline: New options to accomodate the nusplitsc and nucalcsaa
tasks. Added optional call to the new 'nusplitsc' module (parameter
'runsplitsc', set to 'no' by default) and modified the call to
'nucalcsaa' from nupipeline.
- nuflagbad: Minor bug fix to exclude hot pixels in the output
badpixel file (nu*_bp.fits) in case of reprocessed files;
- nuscreen: Addition of a new keyword (OBSMODE) storing the NuSTAR
observing mode to the output cleaned event files;
- nufilter: Addition of the keywords MJDREFI and MJDREFF in the
output filter (nu*.mkf) file;
- nucoord: Value update of the RA_NOM and DEC_NOM keywords in the
+0 and +6 extension of unfiltered event files.
--------------------------------------------------------------------------------------------------------------------------------------
********************************************************************************
NUSTAR
********************************************************************************
NuSTARDAS v1.7.0 software patch:
- nucalcsaa: Adjust the behavior of nucalcsaa so that fselect is no
longer run even when saamode='NONE' & tentacle=no. This resolves
the known issues where nupipeline was crashing when the 01 and 02
event files were empty (mostly when looking at the Sun).
- nupipeline: (1) nupipeline now selects an appropriate TLE file
from the auxil directory instead of simply exiting with an error
message. (2) Corrected a bug in the input parser that occurred
when parameter 'statusexpr' was not set to 'DEFAULT'.
- nuscreen: Correction of a bug when input parameter 'statusexpr'
is not set to DEFAULT.
NuSTARDAS v1.7.1 software patch:
- nufilter: added the parameter 'timeadj=DEFAULT when running the
multi-mission FTOOL prefilter.
--------------------------------------------------------------------------------------------------------------------------------------
********************************************************************************
NUSTAR
********************************************************************************
NuSTARDAS v1.8.0 software patch:
Note that the CALDB patch 20170727 is required when using the new 'saacalc=3' mode.
- nucalcsaa:
Add new 'saacalc=3' operation mode. This accounts for the changes in the
internal background related to the evolution of the South Atlantic Anomaly
(SAA) throughout the solar cycle. This improves on the 'saacalc=2', though
users should check the background filtering reports provided by the NuSTAR
SOC (http://www.nustar.caltech.edu/page/background) when deciding which
filter parameters to use.
Improve handling of input cleaned event files. This patch adjusts the
behavior of nucalcsaa so that the fselect FTOOL is no longer run if the
SHIELD column is not present in the cleaned event file. The SHIELD column
is truncated when the 'cleancols=no' keyword is used by nupipeline, which
is the default case.
- nuexpomap:
Improve the evaluation method for calculating the time-averaged optical
axis location. Before the change, the brightest point on the vignetting-
corrected exposure map could be off from the time-averaged optical axis
position by up to 10". This has been fixed and the two locations will
now correspond to each other. This change affects only the vignetting
corrected exposure map, which is used in the light curve generation.
It has no impact on the ARF and RMF generation.
- nuscreen:
Suppress warning messages that occur when a particular event mode does
not contain any exposure time and therefore its associated cleaned event
file is not generated. This commonly occurs for several of the available
data modes (i.e., calibration source data, slew data, and/or data
obtained during the SAA passages).
- numkarf:
Improve effective area calculation for the extended sources in the
'flatflag=yes' case. In the 'flat' ARF case, where the effective area
is not weighted by the observed count rate, the effective areas were
before evaluated with the same weight in each box that covered the
extraction region, regardless of how much area of each box was actually
contained in the extraction region. This was particularly sensitive to
gaps between the CZT detectors that were inside the boxes but outside
the extraction region. To obtain a more accurate effective area, we
now weigh the effective areas with the covering region of each box.
This change only influences the 'flatflag=yes' case.
- nupipeline:
Now supports the keywords for the handling of the new case 'saacalc=3'
of the module nucalcsaa.
--------------------------------------------------------------------------------------------------------------------------------------
********************************************************************************
NUSTAR
********************************************************************************
NuSTARDAS V1.9.0:
- This is primarily a quality-of-life upgrade for the NuSTARDAS.
The two main additions are: (1) Add the ability to save the
barycenter-corrected event files in the output directory when
barycorr=yes. Previously the user had to use cleanup=no to retail the
barycenter-corrected event file, which also stored a number of other
intermediate files. This allows for a more streamlined analysis
approach for timing observations; (2) The "PRIOR" column is now
preserved by default when producing the cleaned event files. This
column is useful for recovering the time-dependent livetime on
sub-second timescales and for high-rate observations. Please see
the NuSTAR Software User's Guide for more details.
There are also several minor bug fixes and other improvements:
- nupipeline: 1) when barycorr=yes and existage=3: updated the
'nuproducts' run setting the orbit file to the "attorb" file;
2) updated the 'nuproducts' run with `write_baryevtfile=no'.
- nuproducts: 1) added new input parameter `write_baryevtfile' to
optionally save the barycenter-corrected event file when barycorr=yes;
2) added an internal check to make sure that pilow, pihigh, pilowarf,
pihigharf, grppibadhigh, grppibadlow are integer numbers; 3) added an
internal gunzip of input mastaspectfile (to speed-up processing).
- nuscreen: added the "PRIOR" column to the list of columns that are
preserved by default when the cleaned event files are created.
--------------------------------------------------------------------------------------------------------------------------------------
********************************************************************************
NUSTAR
********************************************************************************
NuSTARDAS V1.9.1 and V1.9.2:
numetrology:
This NuSTARDAS release is required for analysis of all NuSTAR
observations performed after March 16, 2020. On that date the on-board
metrology system (which tracks the relative motion of the NuSTAR optics
bench and the focal plane bench) underwent a flight software upgrade to
improve the longevity of the metrology laser system.
NuSTARDAS V1.9.1 and V1.9.2 are upgrades to handle metrology data in
METSTATE 4 and 6, respectively, which are flags in the data stream that
self-document the state of the laser metrology system. These software
changes are transparent to the user requiring no syntax changes when
calling Stage 1 of nupipeline. There is no change to the quality of the
science data.
Please note that attempting to use NuSTARDAS V1.9.0 (released with
HEASoft 6.27) or any earlier versions of the NuSTARDAS with data taken
after March 16, 2020 will result in an error during Stage 1 of the
pipeline processing and result in empty PSD, corrected PSD, and mast
files.
This version of NuSTARDAS is backward compatible with all NuSTAR data
in the HEASARC from before March 16, 2020.
More details can be found in the NuSTAR Software Users's Guide.
--------------------------------------------------------------------------------------------------------------------------------------
********************************************************************************
NUSTAR
********************************************************************************
NuSTARDAS v2.0.0:
Requirements: NuSTAR CALDB 20200YXX
This NuSTARDAS implements a new temperature-dependence during the ARF
generation that should improve performance in a small number of cases
for FPMA. The software change is required to account for cases where
the optic for FPMA is "cold", implying a decrease in the effectiveness
of the thermal blanketing (MLI) and a corresponding increase in the
low-energy transmission. This primarily affects observations after 2018.
This builds on the long-term ARF changes for FPMA documented here:
https://heasarc.gsfc.nasa.gov/docs/heasarc/caldb/nustar/docs/release_20200429.txt
The accompany CALDB patch 20200YXX contains the temperature-dependent
corrections and is required for this software version.
Details on the issue can be found in this paper:
https://arxiv.org/abs/2005.00569
More details about the MLI correction can also be found here:
http://www.srl.caltech.edu/NuSTAR_Public/NuSTAROperationSite/mli.php
:: Note ::
This patch requires access to the optics bench housekeeping file
(obebhkfile). This file is distributed for each observation in the
"hk" subdirectory from the HEASARC. If an observation is downloaded
from the HEASARC, as always, we recommend running nupipeline to
recalibrated the data using the latest CALDB files. This will copy
the OBEB HK file into the output directory (commonly the "event_cl"
directory). No further action is required by the user as these input
files are automatically detected by nuproducts during later analysis
if they are present in the event_cl directory.
To call nuproducts directly on the data downloaded from HEASARC
requires the use of new input keyword: `inobebhkfile'. Details of
this are described in the software user's guide
https://heasarc.gsfc.nasa.gov/docs/nustar/analysis/nustar_swguide.pdf
and in the quickstart guide:
https://heasarc.gsfc.nasa.gov/docs/nustar/analysis/nustar_quickstart_guide.pdf
:: Software changes ::
The major changes introduced in this version are:
- numkarf: 1) Added temperature dependency for ARF files CALDB query;
2) Added new 'obebhkfile' input parameter and computation of
"OPT0_5TEMP" average value.
- nuproducts: 1) Modified call of numkarf task; 2) Added new
'obebhkfile' input parameter.
- nupipeline: 1) Modified call of nuproducts task; 2) Added new
`inobebhkfile' and `obebhkfile' input parameters.
- nu_highfits: Modified to support TFORM values '1PD(X)/1PI(X)'.