********************************************************************************
RELEASE NOTES FOR HEASOFT 6.29
July 19, 2021
********************************************************************************
The HEASoft 6.29 release is driven primarily by updates to mission-
specific data analysis software (NICER, NuSTAR, Swift), but includes
other enhancements and fixes.
As before, configuration-related changes throughout the package mean that
users who already have HEASoft installed will need to completely replace
their software installation; there is no clean way to simply overlay
this new release or to update an existing installation.
********************************************************************************
CFITSIO
********************************************************************************
CFITSIO v4.0.0:
- Removed separate directory for zlib/gzip code, and updated
configuration to check for zlib on the user's system (required).
When use of cURL is enabled, it may also pull in zlib such
that user applications may not need to link with it separately.
- Changed version numbering to 3-field format.
- Added new calculator functions SETNULL(x,y) to allow substitution
of NULL values into tables, and GTIOVERLAP() for calculating the
amount of GTI overlap exposure for a time bin.
- Fix added for proper handling of string columns with zero repeat
count.
- Fix to column filtering expressions which write #NULL values to
columns of type (J) format.
- Fix to memory clearing when using polygon shapes in region files.
- Fix to fits_str2time function so that it now flags a particular
case of bad syntax which was previously getting through.
- In ffgclb and ffpclb (read/write byte columns), the "undocumented"
feature of being able to transfer columns 'A' string columnss as
byte arrays is now handled correctly, with improved error checking
via updates to ffgcprll. More documentation on string handling is
in cfitsio.tex.
- Fixed bug in 'colfilter' functionality. When performing a column
deletion of the form -COLNAM*, and multiple matches existed, none
of the matches got deleted. Now the first is deleted as expected.
- Removed separate directory for zlib/gzip code, and updated
configuration to check for zlib on the user's system (required).
- Improved handling of corner case in ffpkn functions.
- In ffgky, modified TULONG case to allow it to read unsigned
values greater than the 8-byte signed limit.
- Fix to parsing of corner case of extended file syntax.
- Major updates to CMake configuration.
- Updated FITSIO Perl module (Astro-FITS-CFITSIO) to v1.15.
********************************************************************************
CCFITS
********************************************************************************
CCFITS v2.6:
- For compatibility with C++17, removed dependencies on
std::binary_function.
- The internal HDU getScaling function is not properly handling the
case where an image of unsigned integer type has a BZERO keyword
but not a BSCALE. The result is it wrongly assumes the image
is a signed integer type.
- The keyword categories hardcoded and returned by the
HDU::keywordCategories() function now apply only to the
HDU::copyAllKeys() function. HDU::readAllKeys() is hardcoded
to a separate (and larger) list of categories. This is to allow
readAllKeys() to retain its previous behavior, while copyAllKeys()
default categories have been narrowed.
- Fixes to updating of Columns' internal buffers for certain cases
of writes or insertions which add rows to a table. Also includes a
speed increase to internal Column write function.
********************************************************************************
HEASP
********************************************************************************
- General:
* Added SPtrimString function to remove leading and trailing whitespace.
Used this in checkCompatibility method for arf, pha, and rmf to avoid
incorrect failure flagging due to leading or trailing blank spaces.
(bug 2.2c)
* Added version of SPwriteKey to write to the primary header and modified
SPwriteKey for string values so it automatically uses the CONTINUE
convention if the string is more than 68 characters.
- grouping:
* Added optimal binning with minimum counts.
* Fixed loadMin, loadOptimal, and loadOptimalEnergy so that
any channels not included in a bin due to an incomplete bin at the end
of the channels to be grouped are given GROUPING=1 and QUALITY=2. This
is consistent with the behaviour of grppha.
- pha:
* Fixed error in rebinChannels in the way that the new quality
vector is calculated if the grouping included elements with bad
quality.
- rmf:
* Fix to rebinChannels. For EBOUNDS arrays that are grouped AND in
descending order, must switch the FirstEltMode/LastEltMode flags
when calling GroupBin for m_channelLowEnergy and m_channelHighEnergy.
(bug 2.2a)
* Fix to write the EBOUNDS HUDVERS as 1.2.0.
* Fixed bug in rebinEnergies: in the case where the rmf is REDIST the
response should be averaged over the binned energies, not summed.
(bug 2.2b)
* Modified uncompress so it does not change the value of the LO_THRES
keyword. Also changed so that uncompressed data are written as
fixed-length vectors.
* Fixed checkCompatibility over-strictness and improved diagnostic
messages. (bug 2.2d)
- table:
* Added an m_Units member to the tableParameter class.
* Added m_FiltExps to the table class to allow multiple model spectra
per parameter grid point. The model spectrum used is selected by
comparing the table filter expression with that in the data set.
* In getValues function, removed the section that selects a subset
of the tableEnergies array based on overlap with the input energies.
This is incompatible with external usages, such as Xspec's
OGIPTable::energyWeights() function, which assume they are using the
full tableEnergies array.
* Replaced addKey calls with SPwriteKey.
********************************************************************************
ATTITUDE
********************************************************************************
- cornymmik (NEW): For calculating perturbed geomagnetic cutoff rigidity
- geomagterp: Corrected usage of SimpleFITS open() function;
SimpleFITS.pm: Fixed bug when writing partial FITS table rows
- prefilter:
New parameters 'orbpropagate' and 'outpropagate' allow prefilter to
extrapolate and interpolate orbits through time.
New parameter 'tlookback' allows prefilter to look back in time for
history-sensitive events.
SAA_TIME now does not increment when 'in' SAA.
Fixed bug in LOCAL_TIME; previous value was totally wrong
New columns SUN_ELV, SUN_TIME (not selected by default ALL setting).
Added BETA_ANGLE and TIME_SINCE_SUNRISE; changed name of SUN_TIME
to TIME_SINCE_SUNSET.
Added MOON_SIZE, MOON_PHASE, MOON_ILLUM output quantities.
Added trapped proton and electron models (AE8MIN, AE8MAX, AP8MIN,
AP8MAX).
Added verbose output that indicates when estimated TLE is done.
xteorbit avoids doing string compares for every orbit interpolation.
- tristarid: Improved messages to help diagnose/resolve problems
involving WCSTools scat.
********************************************************************************
CALTOOLS
********************************************************************************
CALTOOLS:
- quzcif: Updated to use the callib routine cifsl3(). Added fix
for specifying date and time as a dash (date='-' and time ='-'),
which were previously undocumented but allowed usages. A dash
for the date means that checking of the validity start date is
not done. Provides more explicit error messages for chatter >0.
- udcif: Added a new parameter 'caldbver' which when specified
updates the CALDBVER keyword when writing the new cif; default
value of caldbver=NONE, which means that the caldbver keyword
is written with a blank string value; checks for and deletes
cif.tmp file before trying to write this temporary file; updated
error message when problem copying to cif.tmp; removes cif.tmp
if found.
CALLIB:
- cifsl3(): Replacement for cifsl2(); calls a new callib routine
gtcnam() to print the allowed CAL_CNAMs for the specific mission/
instrument combination if chatter>1 and the CAL_CNAM match fails;
Serialized and improved logic for checking the caldb.indx for
mission, instrument and other specified search parameters.
When a check fails now returns an error message to notify the
user which of the checks have failed.
- calutils.mod(): Moved most of the string length limits for
directories, filenames and other values to the calutils.mod
module; increased string length limits for directories and files.
Added addtolistf, addtolisti to dynamically append elements to
float and integer arrays.
- gtcnam(): New subroutine which reads a caldb.indx file and
returns unique, allowed values of the caldb codename (CAL_CNAM)
which specifies the type of the calibration data.
- cgdr(): Increased directory string length using calutils
- cpthnm(): Converted to f90 and relocated from ftoolslib;
increased allowed length of root
- rcnfig(): Updated limits of cifdir, insdir, config using calutils
- wt_spec(): Increased size of keyword records; prevent extraneous
CONTINUE lines from being copied to output
********************************************************************************
FIMAGE
********************************************************************************
- fimg2tab: Allow images with 1000 or more columns when using
tabletype=BINARY and vector=YES; fixed bugs in handling of swapxy=YES
********************************************************************************
FUTILS
********************************************************************************
- finterp: Added log interpolation option (order=-1);
allow TLOGICAL interpolation to an existing numeric column
- fsumrows:
* Added parameter 'ignorenull' for dealing with null values in input;
* Added new output keyword NSUMROWS, the total number of rows processed
* Updated to handle 64-bit integers
* Bug fix to make clobber=YES if outfile is not a valid FITS file
* Properly reset output TSCAL/TZERO keywords if output is changed to float
* Properly reset output T{L,D}{MIN,MAX} keywords if operation is sum,
since sum will invalidate those min/max keywords.
- fverify: Updated to check usage of the INHERIT keyword:
INHERIT keyword must have a logical value (T or F).
INHERIT can only appear in an extension header (never in primary array).
Primary array must have no data (NAXIS=0).
********************************************************************************
FV
********************************************************************************
- Fixed itk_component 'menubutton' issue causing a crash on Fedora Linux.
********************************************************************************
HEAGEN
********************************************************************************
- ftadjustgti: Fixed bug when copyall=NO
********************************************************************************
HEASARC
********************************************************************************
- extractor: Added new hidden parameter usetpixrsel. If set to yes this uses
the TIMEPIXR keyword value to ensure that event selection is done using the
central time of the frame (equivalent to TIMEPIXR=0.5). The default is no.
Also modified resetting of the output GTIs to frame boundaries. If a frame
could not have its events selected in the intput GTIs due to the time used
in event selection being outside the input GTIs then the frame is not
included in the output GTI.
- fadmap: Added error trapping for 0 index array
- grppha: Increased size of keyword records; for grppha long file
paths were being truncated in the output.
- grppha2: Cleaned up compiler errors and warnings
- mathpha: Increased path lengths and internal variables to support
file names longer than a few characters
********************************************************************************
HEASPTOOLS
********************************************************************************
- ftchkpha: Changed HDUVERS1 to HDUVERS, and added more optionals
to look for
- ftdcmprmf: (NEW) Decompress RMF to use fixed-length FITS columns
- ftgenrsp: Fixed error that FWHM read from file were used as sigmas
- ftgrouppha: Added grouptype=file option which reads the GROUPING
from the spectrum file specified by the templatefile parameter and
uses that; added grouptype=optmin for optimal binning with additional
requirement of minimum number of counts
- rbnrmf (wrapper to ftrbnrmf): Fixed for case of binfile being set.
********************************************************************************
HEATOOLS
********************************************************************************
- ftcmpheap (NEW): Compress the heap in variable-length columns of
a FITS table extension.
- ftlist: Bug fix for unusual case of "0A" string columns. For this
case in ftlist_prtab, it should set veclen to 0 instead of 1
- ftmeld (NEW): Merge (append) rows from multiple input tables into
a single output table, more flexible than ftmerge.
- ftmergesort: Corrected usage of SimpleFITS open() function;
SimpleFITS.pm: Fixed bug when writing partial FITS table rows
- ftverify: Updated to check usage of the INHERIT keyword:
INHERIT keyword must have a logical value (T or F).
INHERIT can only appear in an extension header (not in primary array).
Primary array must have no data (NAXIS=0).
********************************************************************************
XIMAGE
********************************************************************************
- Added support for RADESYS FITS keyword for setting equinox
********************************************************************************
XSELECT
********************************************************************************
- Fixed ordering error in the save command options list and descriptions.
- Fixed bug in FILTER TIME SCC which caused input numbers not to
be recognised.
- Added leading "sh" to run command files, required by newer versions
of the Bourne shell.
- Added support for usetpixrsel logical in the mdb to control the parameter of
the same name in extractor.
********************************************************************************
XSPEC
********************************************************************************
Xspec 12.12.0:
New Features
++++++++++++
- New Models:
* wdem : Power-law differential emission measure.
* grbjet : GRB prompt emission (Farinelli et al., 2021).
* rotpol : Rotate polarization.
* zxipab : Power-law distribution of ionized absorbers.
- Updated Models:
* Table models have been enhanced to allow multiple spectra
to be associated with each parameter grid point. The spectra to be
used can then be selected using the XFLTnnnn keyword values. The
keyword NFLTEXP gives the number of spectra at each grid point and
the keywords XFXPnnnn hold the filter expression to determine
which is used. At the moment the only filter expressions allowed are of
the form "key:value" which match to XFLTnnnn keyword values in the
spectra. The test application for this is spectropolarimetry.
* A new keyword ESCALE can be added to table model files.
If true this automatically adds an Escale parameter to the model which
is used to scale the energy when reading the model spectrum from
the table.
* Table models with no interpolation parameters can be read.
* Table model parameter names containing spaces are modified to use
underbars instead of spaces to avoid problems.
* recorn has been changed from a multiplicative to a mixing
model (this is for internal xspec reasons and should make no
practical difference).
* Mixing models can now be calculated if there is only one
datagroup.
* The creation of new mixing models has been simplified with the
mixsetup program using the mixmodel.dat file to automatically
create interface routines. In addition the MixUtility class now
includes a number of useful methods which reduce the required
knowledge of xspec internals.
- Markov Chain Monte Carlo:
* For the G-W algorithm the initial walkers are now set using
the proposal distribution.
* Added new proposal distributions gauss deltas <value> and
cauchy deltas <value> where <value> is a factor by which
to multiply the parameter deltas.
* Fixed an error in the calculation of the Geweke Convergence diagnostic
which was too large by a factor of sqrt(2).
- Added lognormal, gamma, cauchy, beta, student's t, half-normal, power-law,
isotropic, and shifted lognormal options for Bayesian priors.
- Removed the word normalization from the y-axis of plots. This
will have little effect because the normalization is almost always
one. For the other cases setplot area performs a similar function.
- The show command has new options frozen, linked, tied (the last two
are identical) which print out the appropriate subsets of the parameters.
- An x preceding any xspec command name is interpreted as that
command name (eg xfit can be used for fit). This can be useful if you want
to alias a command name to do something else (eg if you want to
write your own Tcl script for fit which does other things as well as
running xfit).
- For the flux command when consecutive spectra in the
same datagroup have the same energies, all of the spectrum numbers
are now displayed rather than just the first.
- The ConvolutionInLnSpace template routine has been changed to
pass the index into the parameter array for the energy instead of the energy
itself. Also if the input energies are already log spaced then they
are not rebinned onto an internal array.
- Changed installed header file structure.
Fixes
+++++
All bug fixes to v12.11 and v12.11.1 released as patches are included
in v12.12. In addition the following bugs have been fixed.
- External Fortran programs calling Xspec's xsatbl/xsmtbl
functions would crash when Xspec wrote table model diagnostic
output.
- Running nested scripts would cause a misleading error message
to appear if the nested scripts were in different directories.
- The vmcflow model could cause a plotting crash when its
redshift parameter was set to zero.
- plot eff was not working for newly created faked spectra, unless
the spectra were removed and reloaded.
- When using setplot area for datasets whose responses had EBOUNDS
energies listed in descending order (usual for gratings), the plotted
results where the negative of the correct values.
PyXspec:
New Features
++++++++++++
- To help with passing information to a 3rd-party plotting package
(such as matplotlib), several plot retrieval methods have been added.
These are: Plot.labels(), Plot.contourLevels(), and Plot.z().
- The Spectrum object constructor (__init__ function) now takes optional
arguments for specifying background, response, and arf files. This is
useful when you don't want to use the default file names (if any) that
are stored in the Spectrum file.
- Code added for /svg graphics to work when PyXspec is run in Jupyter
notebooks.
- Beginning with Xspec 12.12.0, spectrum and background value arrays
are no longer divided by the area scale. In PyXspec, that carries
over to the Background.values and Spectrum.values attributes.
Fixes
+++++
- In certain cases where errors occur while the ctrl-c signal handler
is active, the Python session will crash when the user attempts to
exit, or a script finishes.
- PyXspec terminates when AllData.fakeit() can't find a response file
and allowPrompting is turned off.
See also:
https://heasarc.gsfc.nasa.gov/lheasoft/xanadu/xspec/issues/issues.html
********************************************************************************
XSTAR
********************************************************************************
XSTAR v2.56f:
- A bug was fixed whereby two different values for the constant used
to convert from eV to A were used. The difference was in the 6th
and 7th significant digit, which turned out to matter in some
rare circumstances.
- New atomic data for elements F,Na,Al,P,K,Cl,Sc,T,V,Cr,Mn,Co,Cu,Zn
based on calculations described in Mendoza et al., 2018A&A...616A..62M
and its antecedents. This results in a significant increase in the
size of the atomic database and the associated data structures.
- Large scale switch from static memory to dynamically allocated memory
in order to account for the growth of the data structures associated
with the atomic database.
- Major restructuring and renaming of subroutines to better describe
their functionality.
********************************************************************************
NICER
********************************************************************************
NICERDAS Version 8 (2021-04-01_V008)
- Additions of NICER response tools nicerarf and nicerrmf, which calculate
ARF (ancillary response file) and RMF (redistribution matrix file)
- Addition of support for new "FPM Selection" system for event files that
allows the user to select or deselect certain NICER detectors for
analysis (see help for nicerl2 and nifpmsel)
- nicerl2 has a new 'tasks' parameter which allows more fine-grained
control over what tasks are performed; this can save time when
calibration is unchanged but the analyst wishes to try a different
screening selection
- Data selection: new default underonly_range=0-500 and new default
overonly_range=0-1.5; new underonly range is based on imminent CALDB
release extending allowed undershoot range; new overonly range is
based on changing geomagnetic conditions
- New task nimkfmerge which can merge multiple filter files (MKF files)
even if the columns do not match exactly
- Filter files: new default columns:
XTI_PNT_JITTER (pointing jitter);
ANG_DIST_{X,Y} (angular offset in instrument X,Y coordinates)
FPM_DEADTIME (per-FPM deadtime);
MPU_LOWMEM_{FIFO,SCI}_DELTA (indication of on-board data overflow);
HV_ON (indicates if bias voltage is enabled)
FPM_SLOW_LLD (current value of low energy threshold setting);
MPU_NOISE20_COUNT (counts below 200 eV);
MEDIAN_UNDERONLY_COUNT (undershoot count rate, median of array);
BETA_ANGLE (orbital beta angle);
TIME_SINCE_SUNSET (time in seconds since beginning of orbit night);
AE8MIN,AP8MIN - AE8/AP8 modeled electron/proton fluxes
- CALDB access: nicerl2 now updates CALDBVER; new task nicaldbver to report
current NICER CALDB release version
- New task niexposum which displays exposure of individual FPMs for
informational purposes
********************************************************************************
NUSTAR
********************************************************************************
NuSTARDAS v2.1.1:
- numkarf: Added new 'mlicorr' input parameter (default='no') to
enable/disable ARF temperature dependent CALDB query
- nupipeline: Modified call of nuproducts task; Added new 'arfmlicorr'
input parameter.
- nuproducts: Modified call of numkarf task; Added new 'arfmlicorr'
input parameter
- Memory allocation increase for the 'stem' variable (various tasks) to
handle high PID values (fixes buffer overflow crashes on Ubuntu Linux).
********************************************************************************
SWIFT
********************************************************************************
BAT:
- Corrected usage of SimpleFITS open() function in batdetmask,
batdrmgen, batgrbproduct, batsurvey, battsplit;
SimpleFITS.pm: Fixed bug when writing partial FITS table rows
UVOT:
- uvotpipeline (NEW): UVOT data analysis script that supports the
processing done at the Swift Data Center and allows user tailoring
of the processing.
* NOTE: Generating useful results requires registering sky images.
The aspect correction software which does this requires access
to a suitable sky reference catalog (nominally USNO B1); it is
possible to configure the software to query USNO B1 over the
web (see catspec help).
- uvotsource:
* Fixed FITS filename when extension selected
* Improved region validation for UVOT photometry/light curves
* If an exposure map is provided, use it to determine exposure in
source and background regions.
* By default source and background exposure must be consistent
within expdeltaf tolerance.
* Updated photometry calculations to handle different exposure in
source and background regions.
* Support skipping exposures which cannot be processed instead of
yielding an error.
* Added parameters: skipbad, skipreason, expdeltaf, forcephot (see
help file)
* (uvotinteg): Add support for MIN, MAX, and FLAG operations
- uvotmaghist:
* Use output parameter to track skipped exposure (skipped parameter)
* Added parameters: skipbad, expdeltaf, forcephot for passing down
to uvotsource (see help)
* Added skipped output parameter for reporting number of skipped
exposures.
- uvotproduct:
* Added parameters: skipbad, expdeltaf, forcephot for passing down
to uvotsource
- uvotimage:
* Added parameters to enhance user control of processing:
badpixlist, flatfile, badpix, flagqual, keepcorr, trackfile
* Optionally create bad pixel maps (badpixlist and badpix parameters)
* Optionally create quality maps (flagqual parameter)
* Determine central shift-and-add offset during each exposure from
aspect following or attitude data (trackfile parameter)
* Optionally keep the modulo-8 noise pattern corrected and flat-
fielded files (keepcorr parameter)
- (uvotexpmap) uvotfakeaf: Only emit AF records with N_FRAMES>0
- uvotbadpix: Added CORNER bad pixel flag; corrected (swapped) bits
corresponding to HOT and COLD pixels. Note that no processing was
specific to HOT vs COLD pixels.
- uvotflagqual: Fixed 81-character infile limit in uvotflagqual.c:
use PIL_LINESIZE instead of FLEN_CARD.
- Calibration.pm: Fixed call to non-existent function in unusual
error condition
- uvotdetect: Add 'errfilt' parameter to filter results
- uvotrawevtimg: Added aberration parameter for applying velocity
aberration
- WCSTools: Harvard Telescope Data Center (TDC) tool 'scat' has
been added to HEASoft for use by the UVOT software.
XRT (XRTDAS 3.6.1):
- xrtfilter: Fixed quzcif calls: specifying a filter on date
(date="NOW") but not time (time="-") was generating errors in
the latest version of quzcif.
- Memory allocation increase for the 'stem' variable (various tasks) to
handle high PID values (fixes buffer overflow crashes on Ubuntu Linux).
********************************************************************************
XTE
********************************************************************************
- Corrected usage of SimpleFITS open() function in pcadeadcalc1,
pcadeadcalc2, pcadeadspect2, pcaextspect2, pcamergeobsids,
pcaprepfile2, pcaprepobsid, proc2pcu;
SimpleFITS.pm: Fixed bug when writing partial FITS table rows
********************************************************************************
MISC.
********************************************************************************
- HEASoftPy, the Python 3 interface to HEASoft tasks, is now included
in all downloads.
- Automated macOS ARM Perl module fix (compiler choice overrides)
previously distributed as a patch.
- Added configure option to disable inclusion of lynx (--without-lynx),
as lynx is not required in order to use fhelp; users can run
"fhelp <task> online=yes" or "fhelp <task> local=yes" instead.
- fhelp: Fixed case of 'attitude' package vs. task.
- Improved support for Apple clang/clang++ v12; remains unsupported
for MAXI, Suzaku, and XIMAGE.
- Updated readline library to v8.0:
http://tiswww.cwru.edu/php/chet/readline/CHANGES
- Updated WCSLIB to v7.3.1:
https://www.atnf.csiro.au/people/mcalabre/WCS/CHANGES
- PGPLOT SVG driver: Fixed seg fault when returning to use /svg
after changing to another device. Changed closing to </svg>
from </SVG> to match opening. Replaced HTML entities for symbols
by their Unicode equivalents because the HTML entities are not
part of the SVG language standard and are not interpreted by
readers such as IPython.display(); fixed error when defining
font-weight in clip-path.
- Updated PGPLOT-perl (v2.24) and dependency ExtUtils-F77 (v1.24)