******************************************************************************** RELEASE NOTES FOR HEASOFT 6.27 March 26, 2020 ******************************************************************************** The HEASoft 6.27 release is driven primarily by updates to mission- specific data analysis software (MAXI, NICER, NuSTAR, Swift, XTE et al.), but includes many other enhancements and fixes. As before, configuration-related changes throughout the package mean that users who already have HEASoft installed will need to completely replace their software installation; there is no clean way to simply overlay this new release or to update an existing installation. ******************************************************************************** CFITSIO ******************************************************************************** Version 3.48: - Now can handle parentheses in path names rather than automatically interpreting them as output file specifiers. - Fixed bug in imcompress.c that wasn't properly handling conversion between float and double types when reading from a gzip compressed float or double image. - Fixed bug that was preventing use of bracket and parentheses symbols in pathnames when opening multiple READWRITE files, even when requesting no-extended-syntax usage. *This fix necessitates a library interface version number change. - Fixed bug in ffmnhd / fits_movnam_hdu to properly handle wildcard syntax. - Fixed bug in fits_open_extlist to handle filename[EXT] syntax properly. The hdutype parameter may now be null. More documentaion for this function is in cfitsio.tex. - Added new function fits_copy_hdutab to create a new table with the same structure as an existing table. - fits_copy_col / ffcpcl handles long long integer data types more natively to prevent precision loss. - histo.c routines now recognize integer columns that have been scaled by TSCALn keywords and may be closer to floating point type. - Added backward compatibility for very old Rice compressed files which were not using the ZVAL2 keyword in the way that later became standard. - Change made to cfitsio.pc.in to prevent forcing downstream libraries to link against cfitsio's dependencies when using pkgconfig. ******************************************************************************** CCFITS ******************************************************************************** - Converted all cases of catching std::bad_cast exceptions from by-value to by-reference. ******************************************************************************** HEASP ******************************************************************************** General: Made arf, pha, and rmf classes a bit more tolerant by allowing reading of files where extensions are identified only by HDUCLAS* keywords. Replaced deprecated auto_ptr by unique_ptr. Added SPneedVecCol routine to check whether a column should be vector or scalar. Changed SPcopyKeys so it only copies keywords that do not already exist in the target extension. This prevents overwriting keyword values that have been set by the tool calling the routine. grouping: Changed the grouping class to include separate grouping and quality vectors instead of combining them in a single flag vector. Made corresponding changes to methods. Fixed bug which was incorrectly rejecting binning factors of -1 in a binning text file. pha: Fixed a bug in convertUnits() which prevented the conversion working in the case of input fluxes in ph/cm^2/s/keV In rebinChannels set the grouping identically to 0 rather than 1 for consistency with the standard since a GROUPING keyword will be written. Changed setting of output quality to be bad if either any input channel making the bin is bad or if phafile grouping is being used any channel in the grouping pha is bad. arf: Added handy routines to return the extension numbers of SPECRESP extensions in a file and the type of an extension. arfII: Made recognition of arf extensions more flexible in NumberofARFs. rmf: When writing output files change N_GRP and CHANNEL to J format if required. Fixed out-of-bounds writes in cases when the F_CHAN and N_CHAN vector columns are longer than N_GRP. Modified rmf::read so that the EBOUNDS data is first read into a separate rmf object so it can test for consistency between MATRIX and EBOUNDS keywords such as CHANTYPE and INSTRUME. If inconsistencies are found then messages are written to SPreportError so the calling program can check for them. The keywords from MATRIX take priority over those from EBOUNDS. Add handy functions to return vectors of extension numbers of EBOUNDS and MATRIX extensions. Changed rmf compression criterion to only include elements with values > LO_THRES instead of >= LO_THRES. Write EBOUNDS and MATRIX energy columns as D instead of E to avoid precision issues with high-resolution optical/UV spectra. table: Changed ENERGIES columns from single to double precision. Fixed a bug in convertUnits() which prevented the conversion working in the case of input fluxes in ph/cm^2/s/keV ******************************************************************************** ATTITUDE ******************************************************************************** - prefilter: Updated to support POSITION and VELOCITY inputs; accepts units of m and km (m/s and km/s) Incorporated the new magnetic field model of IGRF v13 (released December 2019); going back to the year 2000, use full degree-13 precision of published model instead of degree-10 truncation; differences expected to be less than 0.01% Prepared for configurable columns=ALLV4 without losing backward compatibility Added XYZMAGAUX subroutine which computes extra magnetic quantities Added new derived columns relating to magnetic field, view directions and earth viewing; these are optionally produced if columns=ALLV4 (however, compatibility with prefilter v3 is maintained by default): MAGFIELD MAGFIELD_MIN MAGVECT MAGVECT_POL ZENITH_ANGLE EAST_ANGLE NORTH_ANGLE BEARING_ANGLE EARTHPNT_LAT EARTHPNT_LON EARTHPNT_RANGE Fixed bug in dependency ordering which led to segfault; now derived->coordquat is initialized with pointing instead of RA/DEC/ROLL Added MAG_ANGLE = angle between pointing direction and magnetic field Added more powerful ability to include or exclude columns from the selected list Support for attfile == NONE for HaloSat (ATT in .orb file) Addition of LOCAL_TIME derived column - geomagterp (new task): New tool for interpolating geomagnetic quantities; the CALDB portion is not ready yet, so for now it's basically a glorified finterp - tristarid: Updated TDISP keywords for rates in source catalog - Updated tasks to allow longer filenames ******************************************************************************** FIMAGE ******************************************************************************** - flst2im: (v1.4.1) Added support for columns scaled with non-integer TSCALn keywords ******************************************************************************** FUTILS ******************************************************************************** - fdiff: Added support for TLONGLONG 64-bit integers - finterp: Check for TLONGLONG type 'sortkey2' parameter can be an expression Improved error output and documentation Fixed bug in handling of new sortkey1=(expr) function with tcheck=YES; tcheck is now disabled when expressions are used Relaxed requirements upon matching input and output column properties when they are not needed; removed restrictions on how FITS filenames are specified; the extension number is no longer required if the 'first table extension' is acceptable; added ability to rename and/or change data type of source column names, as documented in the help; preserves compatibilty with existing usage Added more checking of output columns and make sure any Tn properties of existing columns aren't accidentally preserved; Fixed bug in 'copyall' case when writing existing file Fixed bug when column exists in output but is not last column Fixed bug in which column number is used for backward compatibility; allow copyall=COPYBEFORE and copyall=COPYAFTER for more flexibility in transferring input file to output ******************************************************************************** FV ******************************************************************************** - Fixed crash when plotting image tables - Fixed crash stemming from incorrect WCS initialization ******************************************************************************** HEAGEN ******************************************************************************** - barycorr: In the case where TSTART & TSTOP are not in the header, set tstart=tstop=TIMENULLVAL to prevent trying to write these keywords. This problem was triggered on macOS 10.15 when using a non-Xcode compiler suite (MacPorts, Homebrew). (heautils library) HDget_frac_time(): Initialize valuei & valuef to avoid returning an undefined value (to e.g. barycorr) when TSTART & TSTOP are missing Handle Swift UVOT event files and Swift MAGHIST files (output of uvotevtlc) Document (and warn) about .gz input and output files - ftadjustgti: Now uses fits_open_extlist() - xrtraytrace: Fixed bug that used back-side mirror reflectivity in place of precollimator reflectivity in two places; fixed bug that retrieved MINENERG and MAXENERG keywords redundantly after FITS file was closed; fixed bug that truncated at 7 the number of interactions per photon path written to the PATHCODE column in the output events file; this number is now 8, as intended. ******************************************************************************** HEASARC ******************************************************************************** - extractor: Fixed trap of no xcolf/ycolf in w_events. Moved correction of time for TIMEPIXR to after the time filtering. This may make small differences for missions which do not use TIMEPIXR values of 0.5 but does make the filtering consistent with that from ftselect. - fadmap: Path/filename corrections to COS-B/SAS-2 CALDB; corrected date errors introduced by Y2K updates ******************************************************************************** HEASPTOOLS ******************************************************************************** - Fixed all heasptools so that output files starting with an exclamation mark are handled correctly. - ftgcorrmf, ftmarfrmf, ftrgsrmfsmooth, ftrsp2rmfarf, ftstripnegchan: Use SPcopyKeys for the EBOUNDS extension as well as the MATRIX extension. - ftmkrsp: Trapped case of an input energy bin with no response groups. - ftsdss2xsp: Fixed check of PLATEID keyword - it is in the primary header, not the COADD extension. - genrsp (wrapper to ftgenrsp): Fixed conditional parameter prompts - rbnrmf (wrapper to ftrbnrmf): If a binfile has been specified set cmpmode=binfile; revised to reflect ftrbnrmf prompting behavior (also ftrbnrmf help file): always prompt for cmpmode, and use it to determine prompting for binfile & nchan; if an ebinfile is specified, pass ecmpmode=ebinfile; otherwise pass ecmpmode=linear and the ebfact value. ******************************************************************************** HEATOOLS ******************************************************************************** - ftdiff: Check for TLONGLONG data type - ftimgcalc: Use fit_get_eqcoltype to more accurately reflect data type of image (could be 'float-like' if BSCAL is used) - ftmgtime: Now uses fits_open_extlist(); added compatibility layer to support mgtime's idiosyncracies for future mgtime wrapper to ftmgtime - ftsort: Handle TLONGLONG 64-bit integers in input columns; work with non-integer TSCALn values; relocated fits_copy_hdutab to CFITSIO; fixed case of outfile=infile where the specified column doesn't exist - ftleapsec: Factored out leap second retrieval code into its own Perl library (see above) - fitsverify (standalone ftverify variant): Use fits_open_diskfile (via ifdef) rather than fits_open_file per a user request. This allows for file paths with special characters (e.g. brackets) that would otherwise fail; note that this also necessitates omitting the call to fits_parse_rootname (via ifdef). ******************************************************************************** XRONOS ******************************************************************************** - All tasks updated to fix memory faults occurring under recent Debian- based systems (e.g. Ubuntu 18.10 and newer) ******************************************************************************** XSELECT ******************************************************************************** - Fixed bug in cpd command which could cause a confusing error message if immediately preceded by some other set command. - (xsl_filter_intensity) Considerable simplification due to using (new) premax and postmax parameters for maketime. - Minor rearrangement for some variables which were in the wrong common blocks; changed to initialize all global variables with some rearrangement to match up with common block order in xselvar.inc; Removed unnecessary include of xselect.h and added one of cfortran.h; added missing Common_AppInit prototype; replaced call of close by the (correct) call of fclose; changed the READ logical to EVREAD for clarity; these changes fix an error restoring saved sessions. - Added support for HaloSat event files. ******************************************************************************** XSPEC ******************************************************************************** HEASoft 6.27 includes Xspec 12.11: New Models: ismdust - Extinction due to silicate and graphite grains. olivineabs - Extinction due to olivine grains. agnslim - AGN super-Eddington accretion model. zkerrbb - Redshifted version of kerrbb. thcomp - Thermally comptonized continuum convolution model. logconst - Convert normalization to log units. log10con - Convert normalization to log10 units. Updated Models: Using the new 201 tabulated temperatures v3.0.9 AtomDB files for apec models. These CEI files reduce potential interpolation problems (see http://atomdb.org/interpolation/index.php). The interpolation scheme can be switched between log and linear interpolation using xset APECLOGINTERP. Another option to reduce interpolation errors is provided by xset APECUSENEI which switches all calculation of CEI spectra through the NEI code. This is slower but can be used to check calculations. To recover the older v3.0.9 files use xset APECROOT 3.0.9_51. If you do this and also want to use NEI models then you also need to use xset NEIAPECROOT 3.0.9. The diskline, rdblur, kdblur, kdblur2, laor, laor2, kerrconv, kerrdisk have all been sped up using a new FFT-based convolution routine. The xscat model has been updated to extend the range of the Rext parameter for MRN and ZDABAS options from 0-199 arcsec to 0-236 arcsec. Major overhaul to code calculating statistics. Main visible change is that each spectrum can have a different fit statistic but there is only a single test statistic. Added support for responses which have been split into parts with different resolution. This is very effective for responses which have high resolution but long tails down to low energies. In the case of the XRISM Resolve instrument this will provide speed increases of up to an order of magnitude. Note that if a file contains multiple response extensions then XSPEC will read and use them all unless a response extension is specified using e.g. resp.rsp{n}. PyXspec v2.0.3: - Plot values can now be retrieved from the individual additive components within a model, using the new Plot.addComp() method. - Added Fit.nVarPars attribute to perform the equivalent of standard Xspec's "tclout varpar". - Added access to standard Xspec's "rerror" command (for running the "error" command on response parameters. This is now available by passing a second (bool) argument to the Fit.error() method. See also: https://heasarc.gsfc.nasa.gov/lheasoft/xanadu/xspec/issues/issues.html ******************************************************************************** XSTAR ******************************************************************************** Xstar v2.54a: - New in xstar version 2.54a is the inclusion of radiative excitation. The rate of radiative excitation for all bound-bound transitions are calculated and then multiplied by a factor of 1-cfrac. So, if cfrac=1 the effective rate is zero and results from previous versions are recovered. The effects of Thomson scattering as an attenuation process in the continuum have now also been included, and the same 1-cfrac multiplier is used so that previous results can be recovered. ******************************************************************************** ASCA ******************************************************************************** - ascaarf, ascaexpo, ascalin, fmosaic, gisrti, sispi: Updated to use GSL/FGSL random number generator - cleansis, faintdfe: Updated to use GSL/FGSL gamma function and auxiliary routines - ofaintdfe: Removed this deprecated task ******************************************************************************** EXOSAT ******************************************************************************** - tkexomerge: Altered window parenting to make children windows visible ******************************************************************************** MAXI ******************************************************************************** - mxdownload_wget: Added new ATT data types to downloadable products; force gmtime() to use four-digit year - mxextract: Avoid writing repeats of STDGTI rows; replaced printf with headas_printf output ******************************************************************************** NICER ******************************************************************************** NICERDAS v007: - nicerl2: Fixed crash when mkfile is specified but gzipped version does not exist; also changed check for missing $indir, previously reported as a missing file Added parameter 'niprefilter' which re-runs niprefilter task if set to YES (additional new parameters orbfile, attfile, prefilter_columns) - nicerpi, nicertimecal, nicertimeconv, nimaketime, niprefilter, niprefilter2: Changes to fix the timegm() Y2020 "trap": This affects routines that use Perl's Time::LocalTime::timegm() function: http://blogs.perl.org/users/grinnz/2019/07/the-timelocal-trap.html For NICER, which uses timegm() for the year 1970, this would cause problems after the year 2020, as timegm() treats the year entry of "70" as meaning 2070 instead of 1970, when the run date is after the year 2020. - nicerpi: Bubble pigain and pioffset parameters up to 'nicerpi' level - nicertimeconv: Use HeaSoftLeapSec library version of NicerTime.pm; fixed bug in output of GPS time (offset of ~20 sec); better handle leap second transitions; output of MJD - nimpumerge: Pass history parameter to subtasks - niprefilter: When writing NULL columns (missing APID) use same column-ordering as normal; Added ability to re-run prefilter and update .mkf file in place using RUNMODE=UPDATE; Fixed fatal bug in handling of global start/stop time from MPU housekeeping; Fixed bug in handling of vehiclefile=NONE; Be more careful about preserving other extensions of infile; Fixed bug of copying keyword comments when runmode=UPDATE ******************************************************************************** NUSTAR ******************************************************************************** NuSTARDAS V1.9.0: - This is primarily a quality-of-life upgrade for the NuSTARDAS. The two main additions are: (1) Add the ability to save the barycenter-corrected event files in the output directory when barycorr=yes. Previously the user had to use cleanup=no to retail the barycenter-corrected event file, which also stored a number of other intermediate files. This allows for a more streamlined analysis approach for timing observations; (2) The "PRIOR" column is now preserved by default when producing the cleaned event files. This column is useful for recovering the time-dependent livetime on sub-second timescales and for high-rate observations. Please see the NuSTAR Software User's Guide for more details. There are also several minor bug fixes and other improvements: - nupipeline: 1) when barycorr=yes and existage=3: updated the 'nuproducts' run setting the orbit file to the "attorb" file; 2) updated the 'nuproducts' run with `write_baryevtfile=no'. - nuproducts: 1) added new input parameter `write_baryevtfile' to optionally save the barycenter-corrected event file when barycorr=yes; 2) added an internal check to make sure that pilow, pihigh, pilowarf, pihigharf, grppibadhigh, grppibadlow are integer numbers; 3) added an internal gunzip of input mastaspectfile (to speed-up processing). - nuscreen: added the "PRIOR" column to the list of columns that are preserved by default when the cleaned event files are created. ******************************************************************************** ROSAT ******************************************************************************** - detect: Updated to use GSL/FGSL gamma function and auxiliary routines - pcecor, pctcor, pcpicor, pcsasscor: Retired these deprecated tasks; no longer needed since ROSAT data has been reprocessed using the necessary corrections (i.e. with SASS version SASS7_8) ******************************************************************************** SUZAKU ******************************************************************************** - xisarfgen: Updated to support PHAFILE==NONE ******************************************************************************** SWIFT ******************************************************************************** BAT: - batdetmask, bat-burst-advocate, batsurvey, batsurvey-detmask: Changes to fix the timegm() Y2020 "trap": This affects routines that use Perl's Time::LocalTime::timegm() function: http://blogs.perl.org/users/grinnz/2019/07/the-timelocal-trap.html - batcelldetect: MPFIT: Bug fixes for two-sided derivative mode (buffer overflow avoidance as well as correct two-sided calculation) UVOT: - uvotmaghist, uvotproduct, uvotsource: Improved region validation for UVOT photometry/light curves. - uvotproduct: Install uvotproduct ds9 v7 grid file in REFDATA XRT: - xrtgrblc: Modified XRT pipeline with correct settings to match revisions to nH code - xrtgrblc, xrtgrblcspec: Added support for high resolution HI4PI all-sky nH map with nhmap=2 ******************************************************************************** XTE ******************************************************************************** - hxtrsp, pcarsp: Updated to call ftrbnrmf instead of rbnrmf (rbnrmf is merely a wrapper to ftrbnrmf) - pcaextspect2: Changes to fix the timegm() Y2020 "trap": This affects routines that use Perl's Time::LocalTime::timegm() function: http://blogs.perl.org/users/grinnz/2019/07/the-timelocal-trap.html Also: bug fix for C-preprocessor macro expansion error - trans2fits: New wrtparm=O option to create OGIP-compatible event files (mostly cribbed from xenon2fits work several eyars ago); further mods to simplify code and to write new output columns that counts various interesting flags - addshots, addsine, fakelc: Updated to use GSL/FGSL random number generator - xenon2fits: (O)GIP output format now agrees with trans2fits - Removed buggy and erroneous clause meant to deal with empty GTI entries, and other overzealous GTI checking ******************************************************************************** MISC. ******************************************************************************** - Dropped support for Cygwin in favor of Windows Subsystem for Linux (WSL) - Fixed bugs in the DESTDIR mechanism - HEASoft now includes the GSL & FGSL libraries, and therefore GNU Fortran compilers older than v5.x are unsupported. - PGPLOT: Updated PGPLOT dynamic library build to link in PNG library (if available); this aids external packages (e.g. ISIS) using our libpgplot by allowing the PNG dependency to be inherited. SVG driver: improved support for Microsoft Edge SVG implementation - Fixed history keyword writing for long (>72 char) lines; some characters were being lost using the previous algorithm