******************************************************************************** RELEASE NOTES FOR HEASOFT 6.29 July 19, 2021 ******************************************************************************** The HEASoft 6.29 release is driven primarily by updates to mission- specific data analysis software (NICER, NuSTAR, Swift), but includes other enhancements and fixes. As before, configuration-related changes throughout the package mean that users who already have HEASoft installed will need to completely replace their software installation; there is no clean way to simply overlay this new release or to update an existing installation. ******************************************************************************** CFITSIO ******************************************************************************** CFITSIO v4.0.0: - Removed separate directory for zlib/gzip code, and updated configuration to check for zlib on the user's system (required). When use of cURL is enabled, it may also pull in zlib such that user applications may not need to link with it separately. - Changed version numbering to 3-field format. - Added new calculator functions SETNULL(x,y) to allow substitution of NULL values into tables, and GTIOVERLAP() for calculating the amount of GTI overlap exposure for a time bin. - Fix added for proper handling of string columns with zero repeat count. - Fix to column filtering expressions which write #NULL values to columns of type (J) format. - Fix to memory clearing when using polygon shapes in region files. - Fix to fits_str2time function so that it now flags a particular case of bad syntax which was previously getting through. - In ffgclb and ffpclb (read/write byte columns), the "undocumented" feature of being able to transfer columns 'A' string columnss as byte arrays is now handled correctly, with improved error checking via updates to ffgcprll. More documentation on string handling is in cfitsio.tex. - Fixed bug in 'colfilter' functionality. When performing a column deletion of the form -COLNAM*, and multiple matches existed, none of the matches got deleted. Now the first is deleted as expected. - Removed separate directory for zlib/gzip code, and updated configuration to check for zlib on the user's system (required). - Improved handling of corner case in ffpkn functions. - In ffgky, modified TULONG case to allow it to read unsigned values greater than the 8-byte signed limit. - Fix to parsing of corner case of extended file syntax. - Major updates to CMake configuration. - Updated FITSIO Perl module (Astro-FITS-CFITSIO) to v1.15. ******************************************************************************** CCFITS ******************************************************************************** CCFITS v2.6: - For compatibility with C++17, removed dependencies on std::binary_function. - The internal HDU getScaling function is not properly handling the case where an image of unsigned integer type has a BZERO keyword but not a BSCALE. The result is it wrongly assumes the image is a signed integer type. - The keyword categories hardcoded and returned by the HDU::keywordCategories() function now apply only to the HDU::copyAllKeys() function. HDU::readAllKeys() is hardcoded to a separate (and larger) list of categories. This is to allow readAllKeys() to retain its previous behavior, while copyAllKeys() default categories have been narrowed. - Fixes to updating of Columns' internal buffers for certain cases of writes or insertions which add rows to a table. Also includes a speed increase to internal Column write function. ******************************************************************************** HEASP ******************************************************************************** - General: * Added SPtrimString function to remove leading and trailing whitespace. Used this in checkCompatibility method for arf, pha, and rmf to avoid incorrect failure flagging due to leading or trailing blank spaces. (bug 2.2c) * Added version of SPwriteKey to write to the primary header and modified SPwriteKey for string values so it automatically uses the CONTINUE convention if the string is more than 68 characters. - grouping: * Added optimal binning with minimum counts. * Fixed loadMin, loadOptimal, and loadOptimalEnergy so that any channels not included in a bin due to an incomplete bin at the end of the channels to be grouped are given GROUPING=1 and QUALITY=2. This is consistent with the behaviour of grppha. - pha: * Fixed error in rebinChannels in the way that the new quality vector is calculated if the grouping included elements with bad quality. - rmf: * Fix to rebinChannels. For EBOUNDS arrays that are grouped AND in descending order, must switch the FirstEltMode/LastEltMode flags when calling GroupBin for m_channelLowEnergy and m_channelHighEnergy. (bug 2.2a) * Fix to write the EBOUNDS HUDVERS as 1.2.0. * Fixed bug in rebinEnergies: in the case where the rmf is REDIST the response should be averaged over the binned energies, not summed. (bug 2.2b) * Modified uncompress so it does not change the value of the LO_THRES keyword. Also changed so that uncompressed data are written as fixed-length vectors. * Fixed checkCompatibility over-strictness and improved diagnostic messages. (bug 2.2d) - table: * Added an m_Units member to the tableParameter class. * Added m_FiltExps to the table class to allow multiple model spectra per parameter grid point. The model spectrum used is selected by comparing the table filter expression with that in the data set. * In getValues function, removed the section that selects a subset of the tableEnergies array based on overlap with the input energies. This is incompatible with external usages, such as Xspec's OGIPTable::energyWeights() function, which assume they are using the full tableEnergies array. * Replaced addKey calls with SPwriteKey. ******************************************************************************** ATTITUDE ******************************************************************************** - cornymmik (NEW): For calculating perturbed geomagnetic cutoff rigidity - geomagterp: Corrected usage of SimpleFITS open() function; SimpleFITS.pm: Fixed bug when writing partial FITS table rows - prefilter: New parameters 'orbpropagate' and 'outpropagate' allow prefilter to extrapolate and interpolate orbits through time. New parameter 'tlookback' allows prefilter to look back in time for history-sensitive events. SAA_TIME now does not increment when 'in' SAA. Fixed bug in LOCAL_TIME; previous value was totally wrong New columns SUN_ELV, SUN_TIME (not selected by default ALL setting). Added BETA_ANGLE and TIME_SINCE_SUNRISE; changed name of SUN_TIME to TIME_SINCE_SUNSET. Added MOON_SIZE, MOON_PHASE, MOON_ILLUM output quantities. Added trapped proton and electron models (AE8MIN, AE8MAX, AP8MIN, AP8MAX). Added verbose output that indicates when estimated TLE is done. xteorbit avoids doing string compares for every orbit interpolation. - tristarid: Improved messages to help diagnose/resolve problems involving WCSTools scat. ******************************************************************************** CALTOOLS ******************************************************************************** CALTOOLS: - quzcif: Updated to use the callib routine cifsl3(). Added fix for specifying date and time as a dash (date='-' and time ='-'), which were previously undocumented but allowed usages. A dash for the date means that checking of the validity start date is not done. Provides more explicit error messages for chatter >0. - udcif: Added a new parameter 'caldbver' which when specified updates the CALDBVER keyword when writing the new cif; default value of caldbver=NONE, which means that the caldbver keyword is written with a blank string value; checks for and deletes cif.tmp file before trying to write this temporary file; updated error message when problem copying to cif.tmp; removes cif.tmp if found. CALLIB: - cifsl3(): Replacement for cifsl2(); calls a new callib routine gtcnam() to print the allowed CAL_CNAMs for the specific mission/ instrument combination if chatter>1 and the CAL_CNAM match fails; Serialized and improved logic for checking the caldb.indx for mission, instrument and other specified search parameters. When a check fails now returns an error message to notify the user which of the checks have failed. - calutils.mod(): Moved most of the string length limits for directories, filenames and other values to the calutils.mod module; increased string length limits for directories and files. Added addtolistf, addtolisti to dynamically append elements to float and integer arrays. - gtcnam(): New subroutine which reads a caldb.indx file and returns unique, allowed values of the caldb codename (CAL_CNAM) which specifies the type of the calibration data. - cgdr(): Increased directory string length using calutils - cpthnm(): Converted to f90 and relocated from ftoolslib; increased allowed length of root - rcnfig(): Updated limits of cifdir, insdir, config using calutils - wt_spec(): Increased size of keyword records; prevent extraneous CONTINUE lines from being copied to output ******************************************************************************** FIMAGE ******************************************************************************** - fimg2tab: Allow images with 1000 or more columns when using tabletype=BINARY and vector=YES; fixed bugs in handling of swapxy=YES ******************************************************************************** FUTILS ******************************************************************************** - finterp: Added log interpolation option (order=-1); allow TLOGICAL interpolation to an existing numeric column - fsumrows: * Added parameter 'ignorenull' for dealing with null values in input; * Added new output keyword NSUMROWS, the total number of rows processed * Updated to handle 64-bit integers * Bug fix to make clobber=YES if outfile is not a valid FITS file * Properly reset output TSCAL/TZERO keywords if output is changed to float * Properly reset output T{L,D}{MIN,MAX} keywords if operation is sum, since sum will invalidate those min/max keywords. - fverify: Updated to check usage of the INHERIT keyword: INHERIT keyword must have a logical value (T or F). INHERIT can only appear in an extension header (never in primary array). Primary array must have no data (NAXIS=0). ******************************************************************************** FV ******************************************************************************** - Fixed itk_component 'menubutton' issue causing a crash on Fedora Linux. ******************************************************************************** HEAGEN ******************************************************************************** - ftadjustgti: Fixed bug when copyall=NO ******************************************************************************** HEASARC ******************************************************************************** - extractor: Added new hidden parameter usetpixrsel. If set to yes this uses the TIMEPIXR keyword value to ensure that event selection is done using the central time of the frame (equivalent to TIMEPIXR=0.5). The default is no. Also modified resetting of the output GTIs to frame boundaries. If a frame could not have its events selected in the intput GTIs due to the time used in event selection being outside the input GTIs then the frame is not included in the output GTI. - fadmap: Added error trapping for 0 index array - grppha: Increased size of keyword records; for grppha long file paths were being truncated in the output. - grppha2: Cleaned up compiler errors and warnings - mathpha: Increased path lengths and internal variables to support file names longer than a few characters ******************************************************************************** HEASPTOOLS ******************************************************************************** - ftchkpha: Changed HDUVERS1 to HDUVERS, and added more optionals to look for - ftdcmprmf: (NEW) Decompress RMF to use fixed-length FITS columns - ftgenrsp: Fixed error that FWHM read from file were used as sigmas - ftgrouppha: Added grouptype=file option which reads the GROUPING from the spectrum file specified by the templatefile parameter and uses that; added grouptype=optmin for optimal binning with additional requirement of minimum number of counts - rbnrmf (wrapper to ftrbnrmf): Fixed for case of binfile being set. ******************************************************************************** HEATOOLS ******************************************************************************** - ftcmpheap (NEW): Compress the heap in variable-length columns of a FITS table extension. - ftlist: Bug fix for unusual case of "0A" string columns. For this case in ftlist_prtab, it should set veclen to 0 instead of 1 - ftmeld (NEW): Merge (append) rows from multiple input tables into a single output table, more flexible than ftmerge. - ftmergesort: Corrected usage of SimpleFITS open() function; SimpleFITS.pm: Fixed bug when writing partial FITS table rows - ftverify: Updated to check usage of the INHERIT keyword: INHERIT keyword must have a logical value (T or F). INHERIT can only appear in an extension header (not in primary array). Primary array must have no data (NAXIS=0). ******************************************************************************** XIMAGE ******************************************************************************** - Added support for RADESYS FITS keyword for setting equinox ******************************************************************************** XSELECT ******************************************************************************** - Fixed ordering error in the save command options list and descriptions. - Fixed bug in FILTER TIME SCC which caused input numbers not to be recognised. - Added leading "sh" to run command files, required by newer versions of the Bourne shell. - Added support for usetpixrsel logical in the mdb to control the parameter of the same name in extractor. ******************************************************************************** XSPEC ******************************************************************************** Xspec 12.12.0: New Features ++++++++++++ - New Models: * wdem : Power-law differential emission measure. * grbjet : GRB prompt emission (Farinelli et al., 2021). * rotpol : Rotate polarization. * zxipab : Power-law distribution of ionized absorbers. - Updated Models: * Table models have been enhanced to allow multiple spectra to be associated with each parameter grid point. The spectra to be used can then be selected using the XFLTnnnn keyword values. The keyword NFLTEXP gives the number of spectra at each grid point and the keywords XFXPnnnn hold the filter expression to determine which is used. At the moment the only filter expressions allowed are of the form "key:value" which match to XFLTnnnn keyword values in the spectra. The test application for this is spectropolarimetry. * A new keyword ESCALE can be added to table model files. If true this automatically adds an Escale parameter to the model which is used to scale the energy when reading the model spectrum from the table. * Table models with no interpolation parameters can be read. * Table model parameter names containing spaces are modified to use underbars instead of spaces to avoid problems. * recorn has been changed from a multiplicative to a mixing model (this is for internal xspec reasons and should make no practical difference). * Mixing models can now be calculated if there is only one datagroup. * The creation of new mixing models has been simplified with the mixsetup program using the mixmodel.dat file to automatically create interface routines. In addition the MixUtility class now includes a number of useful methods which reduce the required knowledge of xspec internals. - Markov Chain Monte Carlo: * For the G-W algorithm the initial walkers are now set using the proposal distribution. * Added new proposal distributions gauss deltas and cauchy deltas where is a factor by which to multiply the parameter deltas. * Fixed an error in the calculation of the Geweke Convergence diagnostic which was too large by a factor of sqrt(2). - Added lognormal, gamma, cauchy, beta, student's t, half-normal, power-law, isotropic, and shifted lognormal options for Bayesian priors. - Removed the word normalization from the y-axis of plots. This will have little effect because the normalization is almost always one. For the other cases setplot area performs a similar function. - The show command has new options frozen, linked, tied (the last two are identical) which print out the appropriate subsets of the parameters. - An x preceding any xspec command name is interpreted as that command name (eg xfit can be used for fit). This can be useful if you want to alias a command name to do something else (eg if you want to write your own Tcl script for fit which does other things as well as running xfit). - For the flux command when consecutive spectra in the same datagroup have the same energies, all of the spectrum numbers are now displayed rather than just the first. - The ConvolutionInLnSpace template routine has been changed to pass the index into the parameter array for the energy instead of the energy itself. Also if the input energies are already log spaced then they are not rebinned onto an internal array. - Changed installed header file structure. Fixes +++++ All bug fixes to v12.11 and v12.11.1 released as patches are included in v12.12. In addition the following bugs have been fixed. - External Fortran programs calling Xspec's xsatbl/xsmtbl functions would crash when Xspec wrote table model diagnostic output. - Running nested scripts would cause a misleading error message to appear if the nested scripts were in different directories. - The vmcflow model could cause a plotting crash when its redshift parameter was set to zero. - plot eff was not working for newly created faked spectra, unless the spectra were removed and reloaded. - When using setplot area for datasets whose responses had EBOUNDS energies listed in descending order (usual for gratings), the plotted results where the negative of the correct values. PyXspec: New Features ++++++++++++ - To help with passing information to a 3rd-party plotting package (such as matplotlib), several plot retrieval methods have been added. These are: Plot.labels(), Plot.contourLevels(), and Plot.z(). - The Spectrum object constructor (__init__ function) now takes optional arguments for specifying background, response, and arf files. This is useful when you don't want to use the default file names (if any) that are stored in the Spectrum file. - Code added for /svg graphics to work when PyXspec is run in Jupyter notebooks. - Beginning with Xspec 12.12.0, spectrum and background value arrays are no longer divided by the area scale. In PyXspec, that carries over to the Background.values and Spectrum.values attributes. Fixes +++++ - In certain cases where errors occur while the ctrl-c signal handler is active, the Python session will crash when the user attempts to exit, or a script finishes. - PyXspec terminates when AllData.fakeit() can't find a response file and allowPrompting is turned off. See also: https://heasarc.gsfc.nasa.gov/lheasoft/xanadu/xspec/issues/issues.html ******************************************************************************** XSTAR ******************************************************************************** XSTAR v2.56f: - A bug was fixed whereby two different values for the constant used to convert from eV to A were used. The difference was in the 6th and 7th significant digit, which turned out to matter in some rare circumstances. - New atomic data for elements F,Na,Al,P,K,Cl,Sc,T,V,Cr,Mn,Co,Cu,Zn based on calculations described in Mendoza et al., 2018A&A...616A..62M and its antecedents. This results in a significant increase in the size of the atomic database and the associated data structures. - Large scale switch from static memory to dynamically allocated memory in order to account for the growth of the data structures associated with the atomic database. - Major restructuring and renaming of subroutines to better describe their functionality. ******************************************************************************** NICER ******************************************************************************** NICERDAS Version 8 (2021-04-01_V008) - Additions of NICER response tools nicerarf and nicerrmf, which calculate ARF (ancillary response file) and RMF (redistribution matrix file) - Addition of support for new "FPM Selection" system for event files that allows the user to select or deselect certain NICER detectors for analysis (see help for nicerl2 and nifpmsel) - nicerl2 has a new 'tasks' parameter which allows more fine-grained control over what tasks are performed; this can save time when calibration is unchanged but the analyst wishes to try a different screening selection - Data selection: new default underonly_range=0-500 and new default overonly_range=0-1.5; new underonly range is based on imminent CALDB release extending allowed undershoot range; new overonly range is based on changing geomagnetic conditions - New task nimkfmerge which can merge multiple filter files (MKF files) even if the columns do not match exactly - Filter files: new default columns: XTI_PNT_JITTER (pointing jitter); ANG_DIST_{X,Y} (angular offset in instrument X,Y coordinates) FPM_DEADTIME (per-FPM deadtime); MPU_LOWMEM_{FIFO,SCI}_DELTA (indication of on-board data overflow); HV_ON (indicates if bias voltage is enabled) FPM_SLOW_LLD (current value of low energy threshold setting); MPU_NOISE20_COUNT (counts below 200 eV); MEDIAN_UNDERONLY_COUNT (undershoot count rate, median of array); BETA_ANGLE (orbital beta angle); TIME_SINCE_SUNSET (time in seconds since beginning of orbit night); AE8MIN,AP8MIN - AE8/AP8 modeled electron/proton fluxes - CALDB access: nicerl2 now updates CALDBVER; new task nicaldbver to report current NICER CALDB release version - New task niexposum which displays exposure of individual FPMs for informational purposes ******************************************************************************** NUSTAR ******************************************************************************** NuSTARDAS v2.1.1: - numkarf: Added new 'mlicorr' input parameter (default='no') to enable/disable ARF temperature dependent CALDB query - nupipeline: Modified call of nuproducts task; Added new 'arfmlicorr' input parameter. - nuproducts: Modified call of numkarf task; Added new 'arfmlicorr' input parameter - Memory allocation increase for the 'stem' variable (various tasks) to handle high PID values (fixes buffer overflow crashes on Ubuntu Linux). ******************************************************************************** SWIFT ******************************************************************************** BAT: - Corrected usage of SimpleFITS open() function in batdetmask, batdrmgen, batgrbproduct, batsurvey, battsplit; SimpleFITS.pm: Fixed bug when writing partial FITS table rows UVOT: - uvotpipeline (NEW): UVOT data analysis script that supports the processing done at the Swift Data Center and allows user tailoring of the processing. * NOTE: Generating useful results requires registering sky images. The aspect correction software which does this requires access to a suitable sky reference catalog (nominally USNO B1); it is possible to configure the software to query USNO B1 over the web (see catspec help). - uvotsource: * Fixed FITS filename when extension selected * Improved region validation for UVOT photometry/light curves * If an exposure map is provided, use it to determine exposure in source and background regions. * By default source and background exposure must be consistent within expdeltaf tolerance. * Updated photometry calculations to handle different exposure in source and background regions. * Support skipping exposures which cannot be processed instead of yielding an error. * Added parameters: skipbad, skipreason, expdeltaf, forcephot (see help file) * (uvotinteg): Add support for MIN, MAX, and FLAG operations - uvotmaghist: * Use output parameter to track skipped exposure (skipped parameter) * Added parameters: skipbad, expdeltaf, forcephot for passing down to uvotsource (see help) * Added skipped output parameter for reporting number of skipped exposures. - uvotproduct: * Added parameters: skipbad, expdeltaf, forcephot for passing down to uvotsource - uvotimage: * Added parameters to enhance user control of processing: badpixlist, flatfile, badpix, flagqual, keepcorr, trackfile * Optionally create bad pixel maps (badpixlist and badpix parameters) * Optionally create quality maps (flagqual parameter) * Determine central shift-and-add offset during each exposure from aspect following or attitude data (trackfile parameter) * Optionally keep the modulo-8 noise pattern corrected and flat- fielded files (keepcorr parameter) - (uvotexpmap) uvotfakeaf: Only emit AF records with N_FRAMES>0 - uvotbadpix: Added CORNER bad pixel flag; corrected (swapped) bits corresponding to HOT and COLD pixels. Note that no processing was specific to HOT vs COLD pixels. - uvotflagqual: Fixed 81-character infile limit in uvotflagqual.c: use PIL_LINESIZE instead of FLEN_CARD. - Calibration.pm: Fixed call to non-existent function in unusual error condition - uvotdetect: Add 'errfilt' parameter to filter results - uvotrawevtimg: Added aberration parameter for applying velocity aberration - WCSTools: Harvard Telescope Data Center (TDC) tool 'scat' has been added to HEASoft for use by the UVOT software. XRT (XRTDAS 3.6.1): - xrtfilter: Fixed quzcif calls: specifying a filter on date (date="NOW") but not time (time="-") was generating errors in the latest version of quzcif. - Memory allocation increase for the 'stem' variable (various tasks) to handle high PID values (fixes buffer overflow crashes on Ubuntu Linux). ******************************************************************************** XTE ******************************************************************************** - Corrected usage of SimpleFITS open() function in pcadeadcalc1, pcadeadcalc2, pcadeadspect2, pcaextspect2, pcamergeobsids, pcaprepfile2, pcaprepobsid, proc2pcu; SimpleFITS.pm: Fixed bug when writing partial FITS table rows ******************************************************************************** MISC. ******************************************************************************** - HEASoftPy, the Python 3 interface to HEASoft tasks, is now included in all downloads. - Automated macOS ARM Perl module fix (compiler choice overrides) previously distributed as a patch. - Added configure option to disable inclusion of lynx (--without-lynx), as lynx is not required in order to use fhelp; users can run "fhelp online=yes" or "fhelp local=yes" instead. - fhelp: Fixed case of 'attitude' package vs. task. - Improved support for Apple clang/clang++ v12; remains unsupported for MAXI, Suzaku, and XIMAGE. - Updated readline library to v8.0: http://tiswww.cwru.edu/php/chet/readline/CHANGES - Updated WCSLIB to v7.3.1: https://www.atnf.csiro.au/people/mcalabre/WCS/CHANGES - PGPLOT SVG driver: Fixed seg fault when returning to use /svg after changing to another device. Changed closing to from to match opening. Replaced HTML entities for symbols by their Unicode equivalents because the HTML entities are not part of the SVG language standard and are not interpreted by readers such as IPython.display(); fixed error when defining font-weight in clip-path. - Updated PGPLOT-perl (v2.24) and dependency ExtUtils-F77 (v1.24)