SEARCH · Engineering Papers

Results for “Data Science”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

All sources NASA NTRS DOE OSTI

DOE OSTI2022

DevOps and Data: Faster-Time-to-Knowledge through SageOps, MLOps, and DataOps

This report describes the approach, investigation, and prototyping efforts to develop an efficient, reusable methodology and reference framework for applying DevOps to disparate data for data science and data analytics at scale, based on focused application of this methodology and reusable reference framework within Sandia National Laboratories’ Pulsed Power community. Additionally, this report reviews: engineered instantiation of the reference framework used for development and production solutions, our experiences and results in using the reference framework, and future plans regarding research and development.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

An Indicator-based Approach to Sustainable Management of Natural Resources (Chapter 12)

Assessing the sustainability of natural resource management choices for agricultural and forest lands requires quantification of potential changes to a set of environmental and socioeconomic indicators selected to characterize reference scenarios relative to projected future scenarios. Correctly framing the questions with local stakeholders is a critical first step in the sustainability assessment, and the questions that can be addressed are often limited by data availability. Selecting and prioritizing indicators with stakeholders to address their needs and concerns improves the likelihood of investment in monitoring and evaluation of those indicators over time. Computational techniques for analyzing interactions between the selected indicators are inherently affected by the scales and formats of the assembled indicator datasets. Data analytics have the potential to improve understanding of the potential synergies and tradeoffs involved with meeting multiple environmental and socioeconomic goals simultaneously, but timely and appropriate indicator datasets are not always available—even in this new era of “big data.” Continued improvements in data science and data analytics are needed to broaden understanding and acceptance of problems and to provide valuable information for natural resource management. Advances in these areas will enable society to design future landscapes that meet multiple objectives, including the provisioning of agricultural and forest resources along with a variety of ecosystem services (e.g., clean water and healthy soils).

Parish, Esther↗

DOE OSTI2019

Data analytics for designing Fe-9Cr steels (tensile strength)

Extended abstract/paper for 'Data analytics for designing Fe-9Cr steels (tensile strength)' - oral presentation at 2019 AIChE Annual Meeting, Orlando, FL, November 2019; Paper 411c as published in the conference proceedings, section 'Innovations in Methods of Data Science' sponsored by AIChE Applications of Data Science to Molecules and Materials, Computing Systems and Technology Division, and Data and Information Systems.

data science, steel, tensile strength OSTI Subject↗

DOE OSTI2023

Tutorial: Lessons Learned for Behavior Analysts from Data Scientists

Big data is a computing term used to refer to large and complex data sets, typically consisting of terabytes or more of diverse data that is produced rapidly. The analysis of such complex data sets requires advanced analysis techniques with the capacity to identify patterns and abstract meanings from the vast data. The field of data science combines computer science with mathematics/statistics and leverages artificial intelligence, in particular machine learning, to analyze big data. This field holds great promise for behavior analysis, where both clinical and research studies produce large volumes of diverse data at a rapid pace (i.e., big data). This article presents basic lessons for the behavior analytic researchers and clinicians regarding integration of data science into the field of behavior analysis. We provide guidance on how to collect, protect, and process the data, while highlighting the importance of collaborating with data scientists to select a proper machine learning model that aligns with the project goals and develop models with input from human experts. Here, we hope this serves as a guide to support the behavior analysts interested in the field of data science to advance their practice or research, and helps them avoid some common pitfalls.

42 ENGINEERING↗

DOE OSTI2017

The medical science DMZ: a network design pattern for data-intensive medical science

Abstract Objective We describe a detailed solution for maintaining high-capacity, data-intensive network flows (eg, 10, 40, 100 Gbps+) in a scientific, medical context while still adhering to security and privacy laws and regulations. Materials and Methods High-end networking, packet-filter firewalls, network intrusion-detection systems. Results We describe a “Medical Science DMZ” concept as an option for secure, high-volume transport of large, sensitive datasets between research institutions over national research networks, and give 3 detailed descriptions of implemented Medical Science DMZs. Discussion The exponentially increasing amounts of “omics” data, high-quality imaging, and other rapidly growing clinical datasets have resulted in the rise of biomedical research “Big Data.” The storage, analysis, and network resources required to process these data and integrate them into patient diagnoses and treatments have grown to scales that strain the capabilities of academic health centers. Some data are not generated locally and cannot be sustained locally, and shared data repositories such as those provided by the National Library of Medicine, the National Cancer Institute, and international partners such as the European Bioinformatics Institute are rapidly growing. The ability to store and compute using these data must therefore be addressed by a combination of local, national, and industry resources that exchange large datasets. Maintaining data-intensive flows that comply with the Health Insurance Portability and Accountability Act (HIPAA) and other regulations presents a new challenge for biomedical research. We describe a strategy that marries performance and security by borrowing from and redefining the concept of a Science DMZ, a framework that is used in physical sciences and engineering research to manage high-capacity data flows. Conclusion By implementing a Medical Science DMZ architecture, biomedical researchers can leverage the scale provided by high-performance computer and cloud storage facilities and national high-speed research networks while preserving privacy and meeting regulatory requirements.

Peisert, Sean↗

DOE OSTI2023

2022 American Conference on Neutron Scattering (ACNS 2022)

The 11th American Conference on Neutron Scattering (ACNS 2022) will be held on June 5-9, 2022, in Boulder, CO. The Conference will provide essential information on the breadth and depth of current neutron-related research worldwide. Hosted by the Neutron Scattering Society of America, this year’s Conference will feature a combination of invited and contributed talks, poster sessions, and tutorials. Topics of the conference are: Advances in Neutron Facilities, Instrumentation and Software: Developments in sources, instrumentation, sample environments and control software. Hard Condensed Matter: Magnetism, correlated metals, quantum/topological materials, superconductors, ferroelectrics, multiferroics, glasses, and disorder phenomena. Submissions outlining examples of neutron scattering in industrial and engineering applications involving hard condensed matter systems are also encouraged. Soft Matter: Neutron studies of soft materials and related fields including in situ and in operando studies. Polymers, surfactants, emulsions, gels, nanoparticles, colloidal suspensions and more. Submissions of computational studies or applications of machine learning beneficial to neutron scattering experiments, as well as examples of neutron scattering in industrial and engineering applications are strongly encouraged. Biology, Biophysics and Biotechnology: Neutron studies of biological and biologically relevant systems. Proteins, bio membranes, biological assemblies, natural materials, nucleic acids, drug-delivery platforms and biomedical systems. Submissions of computational studies or applications of machine learning beneficial to biological neutron scattering experiments, as well as examples of neutron scattering in applied research involving biological systems, are strongly encouraged. Materials Chemistry and Energy: Neutron-based studies of functional materials and materials for energy applications. Examples include porous materials such as metal organic frameworks (MOFs), zeolites; phosphors; novel pigments; electrolytes; catalysts; ionic conductors/cathode materials; photovoltaic materials (hybrid perovskites); thermoelectrics; magnetocalorics/electrocalorics. Structural Materials and Engineering: Neutron scattering studies of materials and engineering processes including structural materials, concrete and metals, as well as engineering processes including combustion, corrosion, additive manufacturing, and others. Neutron Physics: Fundamental physical studies of the neutron and related areas. Emerging Applications in Neutron Scattering: Machine Learning and Data Science: Advances in computing power have contributed to rapidly evolving machine learning and data science fields that can be leveraged to the benefit of the neutron scattering community. The purpose of this session is to highlight recent advances in machine learning and data science and to serve as the foundation of a parallel data and computation track highlighting computation advances and applications in neutron scattering throughout the conference.

36 MATERIALS SCIENCE↗

DOE OSTI2018

Dynamic Workflows for Routine Materials Discovery in Surface Science

The rising application of informatics and data science tools for studying inorganic crystals and small molecules has revolutionized approaches to materials discovery and driven the development of accurate machine learning structure/property relationships. In this paper, we discuss how informatics tools can accelerate research, and we present various combinations of workflows, databases, and surrogate models in the literature. This paradigm has been slower to infiltrate the catalysis community due to larger configuration spaces, difficulty in describing necessary calculations, and thermodynamic/kinetic quantities that require many interdependent calculations. We present our own informatics tool that uses dynamic dependency graphs to share, organize, and schedule calculations to enable new, flexible research workflows in surface science. This approach is illustrated for the large-scale screening of intermetallic surfaces for electrochemical catalyst activity. Similar approaches will be important to bring the benefits of informatics and data science to surface science research. Lastly, we provide our perspective on when to use these tools and considerations when creating them.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2016

Cosmic shear measurements with Dark Energy Survey Science Verification data

Here, we present measurements of weak gravitational lensing cosmic shear two-point statistics using Dark Energy Survey Science Verification data. We also demonstrate that our results are robust to the choice of shear measurement pipeline, either ngmix or im3shape, and robust to the choice of two-point statistic, including both real and Fourier-space statistics. Our results pass a suite of null tests including tests for B-mode contamination and direct tests for any dependence of the two-point functions on a set of 16 observing conditions and galaxy properties, such as seeing, airmass, galaxy color, galaxy magnitude, etc. We furthermore use a large suite of simulations to compute the covariance matrix of the cosmic shear measurements and assign statistical significance to our null tests. We find that our covariance matrix is consistent with the halo model prediction, indicating that it has the appropriate level of halo sample variance. We also compare the same jackknife procedure applied to the data and the simulations in order to search for additional sources of noise not captured by the simulations. We find no statistically significant extra sources of noise in the data. We determined that the overall detection significance with tomography for our highest source density catalog is 9.7 σ. Cosmological constraints from the measurements in this work are presented in a companion paper [DES et al., Phys. Rev. D 94, 022001 (2016).].

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2018

What is Machine Learning?

Have you ever been unsure what people mean when they talk about 'machine learning'? What machines? And what are they learning? This video is for you! For more information about Machine Learning at LLNL and other Data Science projects, visit the LLNL Data Science Institute website at data-science.llnl.gov

3-D PRINTING↗

DOE OSTI2019

SDN-NGenIA, a Software Defined Next Generation Integrated Architecture for HEP and Data Intensive Science

This project aims to enable the LHC and other leading programs in high energy physics and other global science domains funded by the DOE to operate with a new level of efficiency and control, through the development of a Next Generation Integrated Architecture (NGenIA) based on intelligent software defined network (SDN)-driven network systems coupled to high throughput applications. While the initial focus is on the challenging LHC use case, the systems and products being developed are general, and apply to many fields of data intensive science ranging from astrophysical sky surveys to bioinformatics and earth observation, as well as other organizations facing the challenges of extracting knowledge from distributed multi-petabyte data stores. A central concept in this development program is a new paradigm ‘consistent network operations’ among widely distributed computing and storage facilities, where stable high throughput flows, at set rates, cross load-balanced network paths, up to flexible high water marks that are adjusted in real time to accommodate other network traffic. The large smooth flows are launched and managed by SDN services that act in concert with the experiments’ site-resident data distribution and management systems, to meet the expanding needs of the science programs. The technical goals include the construction of autonomous, intelligent site-resident services that interact dynamically with network-resident services, and with the science programs’ principal data distribution and management tools, to request or command network resources in support of high throughput petascale to exascale workflow Specific work items include: 1. Developing compact Data Transfer Nodes (DTNs) with auto-tuning functions that support data transfer rates in the 100 Gbps to the 1 Tbps range when used with high throughput data transfer applications 2. Deep site orchestration among virtualized clusters, storage subsystems and subnets to successfully coschedule CPU, storage and network resources 3. Science-program designed site architectures, operational modes, and policy and resource usage priorities, adjudicated across multiple network domains and virtual organizations, using the orchestration functions and methods 4. Seamlessly extending end-to-end operation across both extra-site and intra-site boundaries through the use of Open vSwitch (OVS), FireQoS + next generation Science DMZs 5. Novel methods of system integration that enable granular control of extreme scale long distance transfers through flow matching of scattered source-destination address pairs to multi-domain dynamic circuits 6. Funneling massive sets of streams to DTNs at the site edge hosting petascale buffer pools configured for flows of 100 Gbps and up, exploiting state of the art data transfers 7. Adaptive scheduling based on pervasive end-to-end monitoring, including DTN or compute-node resident agents providing comprehensive end-system profiling 8. Developing unsupervised and supervised machine learning and modeling methods to optimize the workflow involving terabyte to multi-petabyte datasets.

47 OTHER INSTRUMENTATION↗

DOE OSTI2017

Weak-lensing mass calibration of redMaPPer galaxy clusters in Dark Energy Survey Science Verification data

We use weak-lensing shear measurements to determine the mean mass of optically selected galaxy clusters in Dark Energy Survey Science Verification data. In a blinded analysis, we split the sample of more than 8,000 redMaPPer clusters into 15 subsets, spanning ranges in the richness parameter $5 \leq \lambda \leq 180$ and redshift $0.2 \leq z \leq 0.8$, and fit the averaged mass density contrast profiles with a model that accounts for seven distinct sources of systematic uncertainty: shear measurement and photometric redshift errors; cluster-member contamination; miscentering; deviations from the NFW halo profile; halo triaxiality; and line-of-sight projections. We combine the inferred cluster masses to estimate the joint scaling relation between mass, richness and redshift, $\mathcal{M}(\lambda,z) \varpropto M_0 \lambda^{F} (1+z)^{G}$. We find $M_0 \equiv \langle M_{200\mathrm{m}}\,|\,\lambda=30,z=0.5\rangle=\left[ 2.35 \pm 0.22\ \rm{(stat)} \pm 0.12\ \rm{(sys)} \right] \cdot 10^{14}\ M_\odot$, with $F = 1.12\,\pm\,0.20\ \rm{(stat)}\, \pm\, 0.06\ \rm{(sys)}$ and $G = 0.18\,\pm\, 0.75\ \rm{(stat)}\, \pm\, 0.24\ \rm{(sys)}$. The amplitude of the mass-richness relation is in excellent agreement with the weak-lensing calibration of redMaPPer clusters in SDSS by Simet et al. (2016) and with the Saro et al. (2015) calibration based on abundance matching of SPT-detected clusters. Our results extend the redshift range over which the mass-richness relation of redMaPPer clusters has been calibrated with weak lensing from $z\leq 0.3$ to $z\leq0.8$. Calibration uncertainties of shear measurements and photometric redshift estimates dominate our systematic error budget and require substantial improvements for forthcoming studies.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2015

Wide-Field Lensing Mass Maps from Dark Energy Survey Science Verification Data

We present a mass map reconstructed from weak gravitational lensing shear measurements over 139 deg 2 from the Dark Energy Survey science verification data. The mass map probes both luminous and dark matter, thus providing a tool for studying cosmology. We also find good agreement between the mass map and the distribution of massive galaxy clusters identified using a red-sequence cluster finder. Potential candidates for superclusters and voids are identified using these maps. We measure the cross-correlation between the mass map and a magnitude-limited foreground galaxy sample and find a detection at the 6.8σ level with 20 arc min smoothing. These measurements are consistent with simulated galaxy catalogs based on N-body simulations from a cold dark matter model with a cosmological constant. This suggests low systematics uncertainties in the map. Finally, we summarize our key findings in this Letter; the detailed methodology and tests for systematics are presented in a companion paper.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2025

47 Tuc in Rubin Data Preview 1. Exploring Early LSST Data and Science Potential

We present analyses of the early data from Rubin Observatory’s Data Preview 1 (DP1) for the field of the globular cluster 47 Tuc. The DP1 data set for 47 Tuc includes four nights of observations from the Rubin Commissioning Camera (LSSTComCam), covering multiple bands (ugriy). We address challenges of crowding in the inner region of the cluster and toward the SMC in DP1, and demonstrate improved star–galaxy separation by fitting fifth-degree polynomials to the stellar loci in color–color diagrams and applying multidimensional sigma clipping. We compile a catalog of 3576 probable 47 Tuc member stars selected via a combination of isochrone, Gaia proper-motion, and color–color space matched filtering. We explore the sources of photometric scatter in the 47 Tuc color–color sequence, evaluating contributions from various potential sources, including differential extinction within the cluster. Finally, of the 72 well-characterized variables in the field, we recover three known variable stars, including two RR Lyrae and one eclipsing binary, in the coadd-based object catalog, and identify 62 in the difference image-based object catalog. Although the DP1 lightcurves have sparse temporal sampling, they appear to follow the patterns of densely sampled literature lightcurves well. Despite some data limitations for crowded-field stellar analysis, DP1 demonstrates the promising scientific potential for future LSST data releases.

Choi, Yumi [NSF National Optical-Infrared Astronom↗

DOE OSTI2016

Integration Of PanDA Workload Management System With Supercomputers for ATLAS and Data Intensive Science

The Large Hadron Collider (LHC), operating at the international CERN Laboratory in Geneva, Switzerland, is leading Big Data driven scientific explorations. Experiments at the LHC explore the fundamental nature of matter and the basic forces that shape our universe, and were recently credited for the discovery of a Higgs boson. ATLAS, one of the largest collaborations ever assembled in the sciences, is at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, the ATLAS experiment is relying on a heterogeneous distributed computational infrastructure. The ATLAS experiment uses PanDA (Production and Data Analysis) Workload Management System for managing the workflow for all data processing on over 150 data centers. Through PanDA, ATLAS physicists see a single computing facility that enables rapid scientific breakthroughs for the experiment, even though the data centers are physically scattered all over the world. While PanDA currently uses more than 250,000 cores with a peak performance of 0.3 petaFLOPS, LHC data taking runs require more resources than Grid computing can possibly provide. To alleviate these challenges, LHC experiments are engaged in an ambitious program to expand the current computing model to include additional resources such as the opportunistic use of supercomputers. We will describe a project aimed at integration of PanDA WMS with supercomputers in United States, Europe and Russia (in particular with Titan supercomputer at Oak Ridge Leadership Computing Facility (OLCF), MIRA supercomputer at Argonne Leadership Computing Facilities (ALCF), Supercomputer at the National Research Center Kurchatov Institute , IT4 in Ostrava and others). Current approach utilizes modified PanDA pilot framework for job submission to the supercomputers batch queues and local data management, with light-weight MPI wrappers to run single threaded workloads in parallel on LCFs multi-core worker nodes. This implementation was tested with a variety of Monte-Carlo workloads on several supercomputing platforms for ALICE and ATLAS experiments and it is in full production for the ATLAS experiment since September 2015. We will present our current accomplishments with running PanDA WMS at supercomputers and demonstrate our ability to use PanDA as a portal independent of the computing facilities infrastructure for High Energy and Nuclear Physics as well as other data-intensive science applications, such as bioinformatics and astro-particle physics.

Data-intensive science↗

DOE OSTI2018

Weak lensing magnification in the Dark Energy Survey Science Verification data

In this paper the effect of weak lensing magnification on galaxy number counts is studied by cross-correlating the positions of two galaxy samples, separated by redshift, using data from the Dark Energy Survey Science Verification dataset. The analysis is carried out for two photometrically-selected galaxy samples, with mean photometric redshifts in the $0.2 < z < 0.4$ and $0.7 < z < 1.0$ ranges, in the riz bands. A signal is detected with a $3.5\sigma$ significance level in each of the bands tested, and is compatible with the magnification predicted by the $\Lambda$CDM model. After an extensive analysis, it cannot be attributed to any known systematic effect. The detection of the magnification signal is robust to estimated uncertainties in the outlier rate of the pho- tometric redshifts, but this will be an important issue for use of photometric redshifts in magnification mesurements from larger samples. In addition to the detection of the magnification signal, a method to select the sample with the maximum signal-to-noise is proposed and validated with data.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2016

Galaxy-galaxy lensing in the Dark Energy Survey Science Verification data

We present galaxy-galaxy lensing results from 139 square degrees of Dark Energy Survey (DES) Science Verification (SV) data. Our lens sample consists of red galaxies, known as redMaGiC, which are specifically selected to have a low photometric redshift error and outlier rate. The lensing measurement has a total signal-to-noise of 29 over scales $0.09 < R < 15$ Mpc/$h$, including all lenses over a wide redshift range $0.2 < z < 0.8$. Dividing the lenses into three redshift bins for this constant moving number density sample, we find no evidence for evolution in the halo mass with redshift. We obtain consistent results for the lensing measurement with two independent shear pipelines, ngmix and im3shape. We perform a number of null tests on the shear and photometric redshift catalogs and quantify resulting systematic uncertainties. Covariances from jackknife subsamples of the data are validated with a suite of 50 mock surveys. The results and systematics checks in this work provide a critical input for future cosmological and galaxy evolution studies with the DES data and redMaGiC galaxy samples. We fit a Halo Occupation Distribution (HOD) model, and demonstrate that our data constrains the mean halo mass of the lens galaxies, despite strong degeneracies between individual HOD parameters.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2022

A proximal trust-region method for nonsmooth optimization with inexact function and gradient evaluations

Many applications require minimizing the sum of smooth and nonsmooth functions. For example, basis pursuit denoising problems in data science require minimizing a measure of data misfit plus an $\ell^1$-regularizer. Similar problems arise in the optimal control of partial differential equations (PDEs) when sparsity of the control is desired. Here, we develop a novel trust-region method to minimize the sum of a smooth nonconvex function and a nonsmooth convex function. Our method is unique in that it permits and systematically controls the use of inexact objective function and derivative evaluations. When using a quadratic Taylor model for the trust-region subproblem, our algorithm is an inexact, matrix-free proximal Newton-type method that permits indefinite Hessians. We prove global convergence of our method in Hilbert space and demonstrate its efficacy on three examples from data science and PDE-constrained optimization.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2018

Aurorasaurus database of real-time, crowd-sourced aurora data for space weather research

This work documents the details of Aurorasaurus citizen science data for the period spanning 2015 and 2016 as well as its routine data filtering protocols. Aurorasaurus citizen science data is a collection of auroral sightings submitted to the project via its website or apps and mined from social media. It is a robust data set and particularly abundant during strong geomagnetic storms when auroral precipitation models have the highest uncertainty. This data is offered to the scientific community for use through an open-access database in its raw and scientific formats, each of which is described in detail in this technical report. Furthermore, by demonstrating its scientific utility, we aim to encourage its integration into auroral research.

54 ENVIRONMENTAL SCIENCES↗