Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data analysis software”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 901 records · Page 50

Collaborative Research: Regional climate-change projections through next-generation empirical and dynamical models (Final Technical Report)

This is the final report for a DOE-funded research project describing the outcome of research on non-homogeneous hidden Markov models (NHMMs) and coupled ocean-atmosphere (O-A) intermediate-complexity models (ICMs) to identify the potentially predictable modes of climate variability, and to investigate their impacts on the regional-scale. The main results consist of extensive development of the hidden Markov models for rainfall simulation and downscaling specifically within the non-stationary climate change context together with the development of parallelized software; application of NHMMs to downscaling of rainfall projections over India; identification and analysis of decadal climate signals in data and models; and, studies of climate variability in terms of the dynamics of atmospheric flow regimes.

58 GEOSCIENCES↗

RTDP: Streaming Readout Real-Time Development and Testing Platform

The Thomas Jefferson National Accelerator Facility (JLab) has created and is currently working on various tools to facilitate streaming readout (SRO) for upcoming experiments. These include reconstruction frameworks with support for Artificial Intelligence/Machine Learning, distributed High Throughput Computing (HTC), and heterogeneous computing which all contribute significantly to swift data processing and analysis. Designing SRO systems that combine such components for new experiments would benefit from a platform that would combine both simulation and execution components for simulation, testing, and validation before large investments are made. The Real-Time Development Platform (RTDP) is being developed as part of an LDRD funded project at JLab. RTDP aims to establish a seamless connection between algorithms, facilitating the seamless processing of data from SRO to analysis, as well as enabling the execution of these algorithms in various configurations on compute and data centers. Individual software components simulating specific hardware can be replaced with actual hardware when it is available.

Gyurjyan, Vardan↗

A Parallel Graph Environment for Real-World Data Analytics Workflows

Economic competitiveness and national security depend increasingly on the insightful analysis of large data sets. The diversity of real-world data sources and analytic workflows impose challenging hardware and software requirements for parallel graph platforms. The irregular nature of graph methods is not supported well by the deep memory hierarchies of conventional distributed systems, requiring new processor and runtime system designs to tolerate memory and synchronization latencies. Moreover, the efficiency of relational table operations and matrix computations are not attainable when data is stored in common graph data structures. In this paper, we present HAGGLE, a high-performance, scalable data analytics platform. The platform’s hybrid data model supports a variety of distributed, thread-safe data structures, parallel programming constructs, and persistent and streaming data. An abstract runtime layer enables us to map the stack to conventional, distributed computer systems with accelerators. The runtime uses multithreading, active messages, and data aggregation to hide memory and synchronization latencies on large-scale systems.

Graph Analytics↗

ARM Data Integrator

The Atmospheric Radiation Measurement (ARM) Data Integrator (ADI) streamlines the development of scientific algorithms and analysis of time-series NetCDF data, and improves the content and consistency of the output data products produced by these algorithms. The framework automates the process of retrieving and preparing data for analysis, and allows users to design output data products through a graphical interface. It also provides a modular, flexible software development architecture that scientists can use to generate C, Python, and IDL source code templates that embed the pre and post processing logic allowing the scientist to focus on only their science. The input data, preprocessing, and output data specifications of algorithms are defined through a graphical interface and stored in a database. ADI implements workflow for data integration and supports user access to data through a library of software modules. Data preprocess capabilities supported include automated retrieval of data from input files, merging the retrieved data into appropriately sized chunks, and transformation of the data onto a common coordinate system grid. Through the graphical interface, users can view the details of both their data products and those in the ARM catalog and allows developers to use existing data product to build new data products. Views of the output data products include an overlay of how the design meets ARM archive's data standards providing the user with a visual cue indicating where their output violates an archive standard. The ADI libraries access the information provided through the GUI via a Postgres database. The ADI framework and its supporting components can significantly decrease the time and cost of implementing scientific algorithms while improving the ability of scientists to disseminate their results.

Source record↗

Malware detection and analysis

Embodiments of the invention describe systems and methods for malicious software detection and analysis. A binary executable comprising obfuscated malware on a host device may be received, and incident data indicating a time when the binary executable was received and identifying processes operating on the host device may be recorded. The binary executable is analyzed via a scalable plurality of execution environments, including one or more non-virtual execution environments and one or more virtual execution environments, to generate runtime data and deobfuscation data attributable to the binary executable. At least some of the runtime data and deobfuscation data attributable to the binary executable is stored in a shared database, while at least some of the incident data is stored in a private, non-shared database.

97 MATHEMATICS AND COMPUTING↗

Closeout of CRADA JSA 2012S004: Chapter 5, Integrated Control System, of the document of the ESS Conceptual Design Report, publicly available at https://europeanspallationsource.se/accelerator-documents

The integrated control system (ICS) is responsible for the whole ESS machine and facility: accelerator, target, neutron scattering instruments and conventional facilities. This unified approach keeps the costs of development, maintenance and support relatively low. ESS has selected a standardised, field-proven controls framework, the Experimental Physics and Industrial Control System (EPICS), which was originally developed jointly by Argonne and Los Alamos National Laboratories. Complementing this selection are best practices and experience from similar facilities regarding platform standardisation, control system development and device integration and commissioning. The components of ICS include the control system core, the control boxes, the BLED database management system, and the human machine interface. The control system core is a set of systems and tools that make it possible for the control system to provide required data, information and services to engineers, operators, physicists and the facility itself. The core components are the timing system that makes possible clock synchronisation across the facility, the machine protection system (MPS) and the personnel protection system (PPS) that prevent damage to the machine and personnel, and a set of control system services. Control boxes are servers that control a collection of equipment (for example a radio frequency cavity). The integrated control system will include many control boxes that can be assigned to one supplier, such as an internal team, a collaborating institute or a commercial vendor. This approach facilitates a clear division of responsibilities and makes integration much easier. A control box is composed of a standardised hardware platform, components, development tools and services. On the top level, it interfaces with the core control system components (timing, MPS, PPS) and with the human-machine interface. At the bottom, it interfaces with the equipment and parts of the facility through a set of analog and digital signals, real-time control loops and other communication buses. The ICS central data management system is named BLED (beam line element databases). BLED is a set of databases, tools and services that is used to store, manage and access data. It holds vital control system configuration and physics-related (lattice) information about the accelerator, target and instruments. It facilitates control system configuration by bringing together direct input-output controller (IOC) con guration and real-time data from proton and neutron beam line models. BLED also simplifies development and speeds up the code-test-debug cycle. The set of tools that access BLED will be tailored to the needs of different categories of users, such as ESS staff physicists, engineers, and operators; external partner laboratories; and visiting experimental instrument users. The human-machine interface is vital to providing a high-quality experience to ICS users. It encompasses a wide array of devices and software tools, from control room screens to engineer terminal windows; from beam physics data tools to post-mortem data analysis tools. It serves users with a wide range of skills from widely varied backgrounds. The Controls Group is developing a set of user profiles to accommodate this diverse range of use-cases and users.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Supply Chain Risk Management: Data Structuring

Supply chain risk management (SCRM) is an area of research that addresses both logistics concepts to maximize efficiency, reliability, and revenue as well as risk features, such as potential weak points, break points, and vulnerabilities within the supply chain. SCRM is used to find risks introduced at each node in a supply chain and how these risks can impact a company’s products, individuals, customers, and reputation. SCRM is a relatively new field, so standardized processes including data structuring are not fully documented. This paper explains the importance of a standard data structuring methodology and how it can enhance current SCRM efforts. Data ingest, structuring, and analysis are predominantly managed by humans. Automating some of the less complex steps can positively impact SCRM by allowing human analysts to focus on more strategic analyses. Types of data to be collected and structured are collected via publicly available information related to hardware, software, and corporate entities. After the data has been collected, the information is formatted in a specific manner, conforming to a schema, to allow for more effective and efficient ingest for further analysis. This paper outlines data structures used by Pacific Northwest National Laboratory for SCRM research and analysis purposes. These structures have been used for hundreds of analyses and have been successful in developing a common baseline. Data structuring is one of the first steps in data standardization, which will further mature and enhance the SCRM research area.

supply chain risk management, data structuring, re↗

Geothermal Reservoir Simulation Results in support of Feasibility Study of Direct District Heating for the Cornell Campus Utilizing Deep Geothermal Energy

This dataset contains input data, code, ReadMe files, output data, and figures that summarize the results of a stochastic analysis of geothermal reservoir production from two potential geothermal reservoirs that were evaluated for the Cornell University Deep Direct-Use project. These potential reservoirs are the Trenton-Black River (TBR) from 2.27-2.3 km depth, and basement rocks from 3.0-3.5 km depth and 3.5-4.0 km depth. Several utilization scenarios consisting of different injection fluid temperatures and flow rates were evaluated for each reservoir. Uncertainty in geologic properties, thermal properties, economic costs, and utilization efficiencies were evaluated using a Monte Carlo analysis of the reservoir simulations. Some reservoir simulations of the TBR were completed using the TOUGH2 software, as implemented in PetraSIM. The PetraSIM run files and associated data are provided with this submission. All other reservoir simulations were completed using the GEOPHIRES software, with some modifications to complete the uncertainty analyses. ReadMe files that describe additions to GEOPHIRES, the GEOPHIRES input data, and the output data are all provided, and references are provided to the code repository. Figures that summarize the reservoir heat production, temperature drawdown, and the probability of meeting targeted building heating demands with the produced heat and fluid temperatures are provided.

15 GEOTHERMAL ENERGY↗

Hybrid methods for cybersecurity analysis :

Early 2010 saw a signi cant change in adversarial techniques aimed at network intrusion: a shift from malware delivered via email attachments toward the use of hidden, embedded hyperlinks to initiate sequences of downloads and interactions with web sites and network servers containing malicious software. Enterprise security groups were well poised and experienced in defending the former attacks, but the new types of attacks were larger in number, more challenging to detect, dynamic in nature, and required the development of new technologies and analytic capabilities. The Hybrid LDRD project was aimed at delivering new capabilities in large-scale data modeling and analysis to enterprise security operators and analysts and understanding the challenges of detection and prevention of emerging cybersecurity threats. Leveraging previous LDRD research e orts and capabilities in large-scale relational data analysis, large-scale discrete data analysis and visualization, and streaming data analysis, new modeling and analysis capabilities were quickly brought to bear on the problems in email phishing and spear phishing attacks in the Sandia enterprise security operational groups at the onset of the Hybrid project. As part of this project, a software development and deployment framework was created within the security analyst work ow tool sets to facilitate the delivery and testing of new capabilities as they became available, and machine learning algorithms were developed to address the challenge of dynamic threats. Furthermore, researchers from the Hybrid project were embedded in the security analyst groups for almost a full year, engaged in daily operational activities and routines, creating an atmosphere of trust and collaboration between the researchers and security personnel. The Hybrid project has altered the way that research ideas can be incorporated into the production environments of Sandias enterprise security groups, reducing time to deployment from months and years to hours and days for the application of new modeling and analysis capabilities to emerging threats. The development and deployment framework has been generalized into the Hybrid Framework and incor- porated into several LDRD, WFO, and DOE/CSL projects and proposals. And most importantly, the Hybrid project has provided Sandia security analysts with new, scalable, extensible analytic capabilities that have resulted in alerts not detectable using their previous work ow tool sets.

Dunlavy, Daniel M.↗

Prototype Data Acquisition and Slow Control Systems for the Mu2e Experiment

The Mu2e experiment at the Fermilab Muon Campus will search for the coherent neutrinoless conversion of a muon into an electron in the field of an aluminum nucleus with a sensitivity improvement by a factor of 10 000 over existing limits. Such a charged lepton flavor-violating reaction probes new physics at a scale unavailable with direct searches at either present or planned high-energy colliders. The Mu2e Trigger and Data Acquisition (TDAQ) system exploits otsdaq as its online Data Acquisition System (DAQ) solution. Furthermore, developed at Fermilab, otsdaq integrates both the artdaq DAQ and the art analysis frameworks for event transfer, filtering, and processing. otsdaq is an online DAQ software suite with a focus on flexibility and scalability and provides a multi-user, web-based, interface accessible through a web browser. The read out controllers (ROCs) stream out zero-suppressed data continuously from the detector subsystems to the data transfer controllers (DTCs). The data stream is then read over the peripheral component interconnect express (PCIe) bus to a software filter algorithm that selects events which are combined with the data flux coming from a cosmic-ray veto (CRV) system. The detector control system (DCS) has been developed using the experimental physics and industrial control system (EPICS) open source platform for monitoring, controlling, alarming, and archiving. The DCS has been integrated into otsdaq. A prototype of the TDAQ system and the DCS has been built at Fermilab's Feynman Computing Center. In this article, we report on the progress of the integration of this prototype in the online otsdaq software.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

A deep learning-guided automated workflow in LipidOz for detailed characterization of fungal fatty acid unsaturation by ozonolysis

Understanding fungal lipid biology and metabolism is critical for antifungal target discovery as lipids play central roles in cellular processes. Nuances in lipid structural differences can significantly impact their functions, making it necessary to characterize lipids in detail to enable and understanding of their roles in these complex systems. In particular, lipid double bond (DB) locations are an important component of lipid structure that can only be determined using a few specialized analytical techniques. Ozone-induced dissociation mass spectrometry (OzID-MS) is one such technique that uses ozone to break lipid DBs, producing pairs of characteristic fragments that allow the determination of DB positions. In this work we apply OzID-MS and LipidOz software to analyze the complex lipids of Saccharomyces cerevisiae yeast strains transfected with different fatty acid desaturases from Histoplasma capsulatum to determine the specific unsaturated lipids produce. The automated data analysis in LipidOz made the determination of DB positions from this large dataset more practical, but manual verification for all targets was still time-consuming. The DL model reduces manual involvement in data analysis, but since it was trained using mammalian lipid extracts, the prediction accuracy on yeast-derived data was reduced. We addressed both shortcomings by retraining the DL model to act as a pre-filter to prioritize targets for automated analysis, providing confident manually verified results but requiring less computational time and manual effort. Our workflow resulted in the determination of novel DB positions and enzymatic specificity.

mass spectrometry, deep learning, Lipidomics, doub↗

Super-resolution model for overlapping peak detection and improved spatial resolution in high-energy diffraction microscopy

Reconstruction quality in Far-field High-Energy Diffraction Microscopy (FF-HEDM) is limited by the spatial resolution of area detectors and the frequent occurrence of overlapping diffraction spots. To address these challenges, we developed a super-resolution (SR) framework using convolutional neural networks (CNNs) to recreate 2D diffraction peaks at up to ×8 resolution from raw detector data. A specialized simulation tool was created to generate synthetic training datasets with varying degrees of peak overlap. Integrated into the Microstructural Imaging using Diffraction Analysis Software (MIDAS), the SR model improves the spatial accuracy and precision of 3D grain reconstruction by an order of magnitude. This approach provides a robust solution for investigation of complex micromechanical states and material classes where the analysis is limited by the presence of overlapping peaks. Furthermore, the methodology developed here can potentially be extended to other techniques that require sub-pixel accuracy for high-fidelity data analysis.

High-energy diffraction microscopy↗

An Integrated Tool for Functional Requirements Analysis, Function Allocation and Task Analysis

The software application is called "HFE-Trace". This is an integrated method and tool for the management of Human Factors Engineering analyses and related data. Its primary purpose is to support the coherent and consistent application of the nuclear industry's best practices for human factors engineering work. The software is a custom Microsoft® Access® application. The application is used (in conjunction with other tools such as spreadsheets, checklists and normal documents where necessary) to collect data on the design of a new nuclear power plant from subject matter experts and other sources. This information is then used to identify potential system and functional breakdowns of the intended power plant design. This information is expanded by developing extensive descriptions of all functions, as well as system performance parameters, operating limits and constraints, and operational conditions. Once these have been verified, the human factors elements are added to each function, including intended operator role, function allocation considerations, prohibited actions, primary task categories, and primary work station. In addition, the application includes a computational method to assess a number of factors such as system and process complexity, workload, environmental conditions, procedures, regulations, etc.) that may shape operator performance. This is a unique methodology based upon principles described in NUREG/CR-3331 ("A methodology for allocating nuclear power plant control functions to human or automatic control") and it results in a semi-quantified allocation of functions to three or more levels of automation for a conceptual automation system. The aggregate of all this information is then linked to the Task Analysis section of the application where the existing information on all operator functions is transformed into task information and ultimately into design requirements for Human-System Interfaces and Control Rooms. This final step includes assessment of methods to prevent potential operator errors.

Hugo, Jacques↗

Using an IIDC/DCAM camera for beam display and analysis

An IIDC/DCAM camera is used to the RHIC beams as they pass through a hydrogen jet. The data is transferred over a 1394 bus to a computer with a Linux operating system. A software process on that computer collects the individual frames and packages them into parameters within the RHIC control system's framework. This information is packaged to continuously update two primary types of clients: One type is responsible for near-real-time display of the images; useful in comfort displays, etc. The second is responsible for collecting data used in analysis of beam dynamics and properties like luminosity.

08 HYDROGEN↗

A Novel Sparse Compositional Technique Reveals Microbial Perturbations

ABSTRACT The central aims of many host or environmental microbiome studies are to elucidate factors associated with microbial community compositions and to relate microbial features to outcomes. However, these aims are often complicated by difficulties stemming from high-dimensionality, non-normality, sparsity, and the compositional nature of microbiome data sets. A key tool in microbiome analysis is beta diversity, defined by the distances between microbial samples. Many different distance metrics have been proposed, all with varying discriminatory power on data with differing characteristics. Here, we propose a compositional beta diversity metric rooted in a centered log-ratio transformation and matrix completion called robust Aitchison PCA. We demonstrate the benefits of compositional transformations upstream of beta diversity calculations through simulations. Additionally, we demonstrate improved effect size, classification accuracy, and robustness to sequencing depth over the current methods on several decreased sample subsets of real microbiome data sets. Finally, we highlight the ability of this new beta diversity metric to retain the feature loadings linked to sample ordinations revealing salient intercommunity niche feature importance. IMPORTANCE By accounting for the sparse compositional nature of microbiome data sets, robust Aitchison PCA can yield high discriminatory power and salient feature ranking between microbial niches. The software to perform this analysis is available under an open-source license and can be obtained at https://github.com/biocore/DEICODE ; additionally, a QIIME 2 plugin is provided to perform this analysis at https://library.qiime2.org/plugins/q2-deicode .

59 BASIC BIOLOGICAL SCIENCES↗

Knowledge management: Role of the the Radiation Safety Information Computational Center (RSICC)

The Radiation Safety Information Computational Center (RSICC) at Oak Ridge National Laboratory (ORNL) is an information analysis center that collects, archives, evaluates, synthesizes and distributes information, data and codes that are used in various nuclear technology applications. RSICC retains more than 2,000 software packages that have been provided by code developers from various federal and international agencies. RSICC’s customers (scientists, engineers, and students from around the world) obtain access to such computing codes (source and/or executable versions) and processed nuclear data files to promote on-going research, to ensure nuclear and radiological safety, and to advance nuclear technology. The role of such information analysis centers is critical for supporting and sustaining nuclear education and training programs both domestically and internationally, as the majority of RSICC’s customers are students attending U.S. universities. Additionally, RSICC operates a secure CLOUD computing system to provide access to sensitive export-controlled modeling and simulation (M&S) tools that support both domestic and international activities. This presentation will provide a general review of RSICC’s activities, services, and systems that support knowledge management and education and training in the nuclear field.

Valentine, Timothy↗

pyEGAF: An open-source Python library for the Evaluated Gamma-ray Activation File

The Evaluated Gamma-ray Activation File (EGAF) is one of the most comprehensive resources for thermal neutron-capture data. This database contains data from prompt gamma activation analysis measurements carried out in a consistent manner using the same experimental configuration at the Budapest Research Reactor for 245 isotopes. Although these valuable datasets have been freely available for many years, one of the drawbacks is the outdated and cryptic Evaluated Nuclear Structure Data File (ENSDF) format that is currently adopted for dissemination, making it difficult for users unfamiliar with the format to access and utilize the data contained therein. Furthermore, the ENSDF format does not readily lend itself to modern computational technologies and a parser is required to interpret the complicated mixed-record format. To help overcome these challenges, we have developed a translator to convert the ENSDF-formatted datasets into an open standard JavaScript Object Notation (JSON) format enabling accessibility to applications using different programming languages running in different environments. To compliment this effort, we have also developed an open-source software package implemented in Python, pyEGAF, that is designed to interact with the JSON data structures for general purpose access, manipulation, and analysis of the neutron-capture $\gamma$-ray data in EGAF. The new format, together with the pyEGAF library, greatly enhances access to the wider applications community where EGAF data may be useful or is required.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗