Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software: data analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

AXEAP : a software package for X-ray emission data analysis using unsupervised machine learning

The Argonne X-ray Emission Analysis Package ( AXEAP ) has been developed to calibrate and process X-ray emission spectroscopy (XES) data collected with a two-dimensional (2D) position-sensitive detector. AXEAP is designed to convert a 2D XES image into an XES spectrum in real time using both calculations and unsupervised machine learning. AXEAP is capable of making this transformation at a rate similar to data collection, allowing real-time comparisons during data collection, reducing the amount of data stored from gigabyte-sized image files to kilobyte-sized text files. With a user-friendly interface, AXEAP includes data processing for non-resonant and resonant XES images from multiple edges and elements. AXEAP is written in MATLAB and can run on common operating systems, including Linux, Windows, and MacOS.

97 MATHEMATICS AND COMPUTING↗

Open Reproducible Electron Microscopy Data Analysis

Electron microscopy (EM) is a cornerstone technique in the materials and biological sciences capable of imaging structures at nano- to atomic-scale resolution. Advances in technologies mean that one acquires datasets at increasing data rates and sizes. These advancements present enormous opportunities for researchers to understand complex systems. However, processing the resulting large-scale, complex data in a reproducible and shareable way is a real challenge for researchers. The building, managing, and maintaining complex workflows in a reproducible manner requires extensive knowledge in several areas outside the researchers’ core skill sets, such as software engineering, data science, and high-performance computing (HPC). Our work demonstrates an innovative approach to solving these problems, enabling reproducible EM data analysis through container encapsulated pipelines. Using modern container technologies, we encapsulate processing elements and connect them using shared memory. We expose user-friendly, advanced algorithms and tools to allow end users to utilize without expert programming skills. The platform enables reproducible, scalable, shareable pipelines for the analysis and visualization of EM data. Focusing on interoperability, we leverage the DOE and other agencies’ existing investments to provide a powerful software platform for EM data analysis.

Harris, Christopher↗

The Analysis Description Language Ecosystem: Latest developments and physics applications

We present latest developments in Analysis Description Language (ADL), a declarative domain-specific language describing the physics algorithm of a HEP data analysis decoupled from software frameworks. Analyses written in ADL can be integrated into any framework for various tasks. ADL is a multipurpose construct with uses ranging from analysis design to preservation, reinterpretation, queries, visualisation, combination, etc. The most advanced infrastructure to execute ADL on events is the CutLang runtime interpreter. Recent technical developments include an automated interface with different data types, generation of the abstract syntax tree, a visualization tool that that auto-converts analysis flows to graphs, incorporation of trained machine learning models and a Jupyter-based plotting tool. We also report physics implications including a large scale LHC analysis implementation and validation effort for beyond the standard model reinterpretation purposes and studies with ATLAS and CMS open data.

Sekmen, Sezen [Kyungpook National Univ., Daegu (Ko↗

MAPPRAISER: A massively parallel map-making framework for multi-kilo pixel CMB experiments

Forthcoming cosmic microwave background (CMB) polarized anisotropy experiments have the potential to revolutionize our understanding of the Universe and fundamental physics. The sought-after, tale-telling signatures will be however distributed over voluminous data sets which these experiments will collect. These data sets will need to be efficiently processed and unwanted contributions due to astrophysical, environmental, and instrumental effects characterized and efficiently mitigated in order to uncover the signatures. This poses a significant challenge to data analysis methods, techniques, and software tools which will not only have to be able to cope with huge volumes of data but to do so with unprecedented precision driven by the demanding science goals posed for the new experiments. A keystone of efficient CMB data analysis is solvers of very large linear systems of equations. Such systems appear in very diverse contexts throughout CMB data analysis pipelines, however they typically display similar algebraic structures and can therefore be solved using similar numerical techniques. Linear systems arising in the so-called map-making problem are one of the most prominent and common ones. In this work we present a massively parallel, flexible and extensible framework, comprised of a numerical library, MIDAPACK, and a high level code, MAPPRAISER, which provide tools for solving efficiently such systems. Here, the framework implements iterative solvers based on conjugate gradient techniques: enlarged and preconditioned using different preconditioners. We demonstrate the framework on simulated examples reflecting basic characteristics of the forthcoming data sets issued by ground-based and satellite-borne instruments, executing it on as many as 16,384 compute cores. The software is developed as an open source project freely available to the community at: https://github.com/B3Dcmb/midapack.

79 ASTRONOMY AND ASTROPHYSICS↗

Developing an Automated Uncertainty Quantification Tool to Improve Watershed-Scale Predictions of Water and Nutrient Cycling

Managing the flow of water, nutrients, and contaminants in watersheds is vital to addressing pressing issues related to water scarcity, access to clean drinking water, energy production, resilience to natural and anthropogenic perturbations, and ecological restoration. Decisions about the management of watersheds critically depend on the accuracy with which the flow of water and chemicals through the watershed can be predicted by computer models. Prediction uncertainty can be reduced by matching the model to data, which are collected in the field at great expense. The contribution of watershed characterization data to reducing uncertainty of relevant model predictions can be evaluated in a so-called data-worth analysis, which provides transparent, quantitative metrics about a data set’s value for the support of relevant watershed management objectives. To achieve this goal, we developed a software package that implements the data-worth analysis approach for use with state-of-the-art watershed models. The purpose of the proposed data-worth analysis is to help decision-makers allocate resources for watershed characterization such that the uncertainty in model predictions can be significantly reduced, which leads to better, more effective management decisions. At the same time, watershed characterization costs can be reduced. The specific technical objectives of this SBIR/STTR Phase II project were to develop a framework and associated software toolsets that implement the uncertainty quantification and data-worth analysis approach for use with state-of-the-art watershed models. This goal was achieved by (A) developing a user-friendly, robust software package that is accessible to a wide audience, including watershed managers, policy-makers, and public stakeholders; (B) by demonstrating application of the prototype on several use cases that are representative of complex watershed management challenges spanning a range of scales and that consider different open-source, DOE-based codes and other modeling platforms; and (C) by gathering information about the needs and requirements from potential users to help guide future developments, ensuring that the final product will be commercially viable. The developed software consists of a graphical user interface that guides the user through a sequence of analysis steps, supported by toolsets that leverage state-of-the-art computational simulation-optimization capabilities. A prototype of the software runs on multiple platforms (PC, Mac, multi-processor Linux environment), is linked to diverse watershed simulators (e.g., ECOSYS, TOUGH2, TOUGHREACT, Amanzi-ATS), performs multiple analysis tasks (predictive simulations, sensitivity analysis, uncertainty analysis, automatic parameter estimation, and data-worth analysis, multicomponent geothermometry), and is readily extensible to include external simulators and analysis tools. The software is being commercialized and will be continually updated to address user needs.

58 GEOSCIENCES↗

MILK : a Python scripting interface to MAUD for automation of Rietveld analysis

Modern diffraction experiments ( e.g. in situ parametric studies) present scientists with many diffraction patterns to analyze. Interactive analyses via graphical user interfaces tend to slow down obtaining quantitative results such as lattice parameters and phase fractions. Furthermore, Rietveld refinement strategies ( i.e. the parameter turn-on-off sequences) tend to be instrument specific or even specific to a given dataset, such that selection of strategies can become a bottleneck for efficient data analysis. Managing multi-histogram datasets such as from multi-bank neutron diffractometers or caked 2D synchrotron data presents additional challenges due to the large number of histogram-specific parameters. To overcome these challenges in the Rietveld software Material Analysis Using Diffraction ( MAUD ), the MAUD Interface Language Kit ( MILK ) is developed along with an updated text batch interface for MAUD . The open-source software MILK is computer-platform independent and is packaged as a Python library that interfaces with MAUD . Using MILK , model selection ( e.g. various texture or peak-broadening models), Rietveld parameter manipulation and distributed parallel batch computing can be performed through a high-level Python interface. A high-level interface enables analysis workflows to be easily programmed, shared and applied to large datasets, and external tools to be integrated with MAUD . Through modification to the MAUD batch interface, plot and data exports have been improved. The resulting hierarchical folders from Rietveld refinements with MILK are compatible with Cinema: Debye–Scherrer , a tool for visualizing and inspecting the results of multi-parameter analyses of large quantities of diffraction data. In this manuscript, the combined Python scripting and visualization capability of MILK is demonstrated with a quantitative texture and phase analysis of data collected at the HIPPO neutron diffractometer.

97 MATHEMATICS AND COMPUTING↗

Mapping structural heterogeneity at the nanoscale with scanning nano-structure electron microscopy (SNEM)

Here, in this work, we explore the use of scanning electron diffraction (also known as 4D-STEM) coupled with electron atomic pair distribution function analysis (ePDF) to understand the local order (structure and chemistry) as a function of position in a complex multicomponent system, a hot rolled, Ni-encapsulated, Zr 65 Cu 17.5 Ni 10 Al 7.5 bulk metallic glass (BMG), with a spatial resolution of 3 nm. We show that it is possible to gain insight into the chemistry and chemical clustering/ordering tendency in different regions of the sample, including in the vicinity of nano-scale crystallites that are identified from virtual dark field images and in heavily deformed regions at the edge of the BMG. In addition to simpler analysis, unsupervised machine learning was used to extract partial PDFs from the material, modeled as a quasi-binary alloy, and map them in space. These maps allowed key insights not only into the local average composition, as validated by EELS, but also a unique insight into chemical short-range ordering tendencies in different regions of the sample during formation. The experiments are straightforward and rapid and, unlike spectroscopic measurements, don’t require energy filters on the instrument. We spatially map different quantities of interest (QoI’s), defined as scalars that can be computed directly from positions and widths of ePDF peaks or parameters refined from fits to the patterns. We developed a flexible and rapid data reduction and analysis software framework that allows experimenters to rapidly explore images of the sample on the basis of different QoI’s. The power and flexibility of this approach are explored and described in detail. Because of the fact that we are getting spatially resolved images of the nanoscale structure obtained from ePDFs we call this approach scanning nano-structure electron microscopy (SNEM), and we believe that it will be powerful and useful extension of current 4D-STEM methods.

36 MATERIALS SCIENCE↗

DRAGraces: A Pipeline for the GRACES High-resolution Spectrograph at Gemini

This paper describes the software Data Reduction and Analysis for GRACES (DRAGraces), which is a pipeline reducing spectra from Gemini Remote Access to the CFHT ESPaDOnS Spectrograph (GRACES) at the Gemini North Telescope. The code is written in the IDL language. It is designed to find all the GRACES frames in a given directory, automatically determine the list of bias, flat, arc, and science frames, and perform the whole reduction and extraction within a few minutes. We compare the output from DRAGraces with that of the Open source Pipeline for ESPaDOnS Reduction and Analysis (OPERA), a pipeline developed at the Canada–France–Hawaii Telescope (CFHT) that also can extract GRACES spectra. Both pipelines were developed completely independently, yet they give very similar extracted spectra. They both have their advantages and disadvantages. For instance, DRAGraces is more straightforward and easy to use and is less likely to be derailed by a parameter that needs to be tweaked, while OPERA offers a more careful extraction that can be significantly superior when the highest resolution is required and when the signal-to-noise ratio is low. One should compare both before deciding which one to use for their science. Yet, both pipelines deliver a fairly comparable resolution power (R ~ 52.8k and 36.6k for DRAGraces and R ~ 58k and 40k for OPERA in high- and low-resolution spectral modes, respectively), wavelength solution, and signal-to-noise ratio per resolution element.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

The Short-Baseline Near Detector at Fermilab: Input to the European Strategy for Particle Physics 2026 Update

SBND is a 112 ton liquid argon time projection chamber (LArTPC) neutrino detector located 110 meters from the Booster Neutrino Beam (BNB) target at Fermilab. Its main goals include searches for eV-scale sterile neutrinos as part of the Short-Baseline Neutrino (SBN) program, other searches for physics beyond the Standard Model, and precision studies of neutrino-argon interactions. In addition, SBND is providing a platform for LArTPC neutrino detector technology development and is an excellent training ground for the international group of scientists and engineers working towards the upcoming flagship Deep Underground Neutrino Experiment (DUNE). SBND began operation in July 2024, and started collecting stable neutrino beam data in December 2024 with an unprecedented rate of ~7,000 neutrino events per day. During its currently approved operation plans (2024-2027), SBND is expected to accumulate nearly 10 million neutrino interactions. The near detector dataset will be instrumental in testing the sterile neutrino hypothesis with unprecedented sensitivity in SBN and in probing signals of beyond the Standard Model physics. It will also be used to significantly advance our understanding of the physics of neutrino-argon interactions ahead of DUNE. After the planned accelerator restart at Fermilab (2029+), opportunities are being explored to operate SBND in antineutrino mode in order to address the scarcity of antineutrino-argon scattering data, or in a dedicated beam-dump mode to significantly enhance sensitivity to searches for new physics. SBND is an international effort, with approximately 40% of institutions from Europe, contributing to detector construction, commissioning, software development, and data analysis. Continued European involvement and leadership are essential during SBND's operations and analysis phase for both the success of SBND, SBN and its role leading up to DUNE.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

A Python Instrument Control and Data Acquisition Suite for Reproducible Research

Tools that standardize and automate experimental data collection are needed for greater confidence in research results. The National Synchrotron Light Source-II (NSLS-II) has generated an open-source Python data acquisition, management, and analysis software suite that automates X-ray experiments and collects an experimental record that facilitates complete reproducibility. Here, we show that the NSLS-II tools are not only useful for X-ray science at large-scale facilities by presenting an add-on package that adapts these tools for use in a small laboratory with common physics and electrical engineering instruments. The composite software suite eases and automates the execution of experiments, records extensive metadata, stores data in portable containers, and speeds up the analysis through tools for comprehensive searches. In total, this software suite increases the reproducibility of laboratory experiments. We demonstrate the software via the evaluation of two lock-in amplifiers-the miniature ADA2200 and the ubiquitous Stanford Research Systems (SRS) SR810. The frequency resolution, signal-to-noise ratio, and dynamic reserve of the lock-in amplifiers are measured and presented. The usage of the software suite is described throughout these measurements so that the reader can implement the tools in their lab.

97 MATHEMATICS AND COMPUTING↗

Simulations of Events for the LUX-ZEPLIN (LZ) Dark Matter Experiment

The LUX-ZEPLIN dark matter search aims to achieve a sensitivity to the WIMP-nucleon spin-independent cross-section down to (1–2) ×10−12 pb at a WIMP mass of 40 GeV/ c 2 . This paper describes the simulations framework that, along with radioactivity measurements, was used to support this projection, and also to provide mock data for validating reconstruction and analysis software. Of particular note are the event generators, which allow us to model the background radiation, and the detector response physics used in the production of raw signals, which can be converted into digitized waveforms similar to data from the operational detector. Inclusion of the detector response allows us to process simulated data using the same analysis routines as developed to process the experimental data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

A New Workflow of X-ray CT Image Processing and Data Analysis of Structural Features in Rock Using Open-Source Software

X-ray computed tomography (CT) images of rock specimens often contain artifacts which must be corrected before scientific analyses are performed. Here, we present a new workflow of automated image processing to utilize poor-quality X-ray CT scan images. The workflow runs on the open-source image analysis software and efficiently separates desired features from low-contrast scanned images. The new workflow is a two-step technique using contrast enhancement and automated feature segmentation to generate noise-free binary images. The results of binary images using the proposed workflow and using a conventional thresholding technique are analyzed to show the quality of the proposed method. The paper also presents a workflow of estimating the structural geometries of features in two and three dimensions. The results of the structural feature analyses and computational time were compared between the open-source (ImageJ) and commercial image analysis software (Bruker Computed Tomography Analyzer). The commercial software was more computationally efficient, but the task-specific macros in open-source software enabled the user-desired automation in image processing and data extraction of desired structural features of comparable quality.

47 OTHER INSTRUMENTATION↗

Decay Curve Correction Analysis Report

The decay curve analysis that is done on the short-lived radionuclide gas samples is used to differentiate between gaseous radionuclides that have the same characteristic gamma decay energy, 511 kiloelectron-volts (keV). A sample of stack gas is isolated and the total counts in the 511 keV peak are counted repeatedly in 10-second intervals to evaluate the decay rate of the sample over time. Analysis of this decay data required a series of steps. First, a raw data report is generated by the gamma acquisition system, based on an analysis template within the acquisition software. The data report file was then loaded into Microsoft Word, and a macro was used to perform minor formatting (remove colons and insert tabs between data columns) to allow analysis within Excel. The file is then saved as a text file at this point. The text file is then uploaded into Excel and a series of macros are used to add labels, calculate radioactive decay constants, and analyze the gamma decay data using linear regression techniques. The analysis template has been used since 1998 for stack 53000303 (TA-53, building 0003, exhaust stack 03) and 2000 for stack 53000702. The overall process, including the gamma report format and the macros used in Word and Excel for processing the report, had remained unchanged until 2015. In October of 2015, staff made a change to the report template in the gamma acquisition software which resulted in an error in the calculations later performed by the Excel macro. This error was not caught until a more in-depth review of the analysis took place regarding 2020 data. This report covers a much more complete review of the issue that occurred regarding the decay curve analysis, a review of the calculations completed to correct the issue, a review of the updated decay curve analysis process, and recommendations for moving forward.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Towards scanning nanostructure X-ray microscopy

This article demonstrates spatial mapping of the local and nanoscale structure of thin film objects using spatially resolved pair distribution function (PDF) analysis of synchrotron X-ray diffraction data. This is exemplified in a lab-on-chip combinatorial array of sample spots containing catalytically interesting nanoparticles deposited from liquid precursors using an ink-jet liquid-handling system. A software implementation is presented of the whole protocol, including an approach for automated data acquisition and analysis using the atomic PDF method. The protocol software can handle semi-automated data reduction, normalization and modeling, with user-defined recipes generating a comprehensive collection of metadata and analysis results. By slicing the collection using included functions, it is possible to build images of different contrast features chosen by the user, giving insights into different aspects of the local structure.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Joint Analysis of Program Data Representations using Machine Learning for Improved Software Assurance and Development Capabilities

We explore the use of multiple deep learning models for detecting flaws in software programs. Current, standard approaches for flaw detection rely on a single representation of a software program (e.g., source code or a program binary). We illustrate that, by using techniques from multimodal deep learning, we can simultaneously leverage multiple representations of software programs to improve flaw detection over single representation analyses. Specifically, we adapt three deep learning models from the multimodal learning literature for use in flaw detection and demonstrate how these models outperform traditional deep learning models. We present results on detecting software flaws using the Juliet Test Suite and Linux Kernel.

97 MATHEMATICS AND COMPUTING↗

Algorithms and file structures to enhance software workflows for ion mobility mass spectrometry (IM-MS): CRADA 410 (Final Report)

This document is the final report for CRADA 410 (Project No. 72496). The purpose of this project was supporting customizations of algorithms and raw data file structures to enhance software workflows for liquid chromatography (LC), mass spectrometry (MS) and ion mobility mass spectrometry (IM-MS)-based metabolite characterization. PNNL worked with Agilent to evaluate and improve the integration of ion mobility into existing MS data analysis methods of Agilent software tools. The project augmented PNNL’s capabilities to analyze complex omics samples. These capabilities are directly beneficial to DOE and PNNL efforts to characterize and analyze compounds in microbial and plant communities. The project assisted Agilent in further developing improved instrument-software solutions combining ion mobility with mass spectrometry for widespread applications in life sciences and other fields.

97 MATHEMATICS AND COMPUTING↗

LipidOz enables automated elucidation of lipid carbon–carbon double bond positions from ozone-induced dissociation mass spectrometry data

Lipids play essential roles in many biological processes and disease pathology, but unambiguous identification of lipids is complicated by the presence of multiple isomeric species differing by fatty acyl chain length, stereospecifically numbered (sn) position, and position/stereochemistry of double bonds. Conventional liquid chromatography-mass spectrometry (LC-MS/MS) analyses enable the determination of fatty acyl chain lengths (and in some cases sn position) and number of double bonds, but not carbon-carbon double bond positions. Ozone-induced dissociation (OzID) is a gas-phase oxidation reaction that produces characteristic fragments from lipids containing double bonds. OzID can be incorporated into ion mobility spectrometry (IMS)-MS instruments for the structural characterization of lipids, including additional isomer separation and confident assignment of double bond positions. The complexity and repetitive nature of OzID data analysis and lack of software tool support have limited the application of OzID for routine lipidomics studies. Here, we present an open-source Python tool, LipidOz, for the automated determination of lipid double bond positions from OzID-IMS-MS data, which employs a combination of traditional automation and deep learning approaches. Our results demonstrate the ability of LipidOz to robustly assign double bond positions for lipid standard mixtures and complex lipid extracts, enabling practical application of OzID for future lipidomics.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗