Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Sources”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 721 records · Page 40

Cyote-attack Chain Estimator

Attack Chain Estimator (ACE) Application Overview The Attack Chain Estimator (ACE) Application is a sophisticated tool designed for the ingestion, classification, sequencing, and enrichment of cybersecurity threat reports. This application leverages advanced machine learning models and extensive historical data to provide comprehensive insights into cyber threats, specifically targeting Industrial Control Systems (ICS). Purpose The primary functions of the ACE Application include: Ingestion of Cybersecurity Threat Reporting: Capable of ingesting text-based threat reports in markdown or text file format. Supports ingestion of structured data from other sources in STIX/JSON format. Classification of Report’s Text-Based Events: Utilizes a DeBERTa classifier, specifically trained on cybersecurity data, to map the events to MITRE ATT&CK for ICS Tactics and Techniques. Classification is performed using multiple Jupyter notebooks and machine learning workflows hosted as FastAPI microservices: regex_data deberta_base_35_train_hft_classifier_mlflow.ipynb hft_regex_classifier_mlflow.ipynb param_train_hft_classifier_mlflow.ipynb regex_tactic_tech.ipynb Ordering of Tactics, Techniques, and Observable Events: Sequences the identified tactics, techniques, and events to form a coherent attack chain. Enrichment with Historical Attack Chain Details: Enhances the attack chain with details from historical attacks using a Markov model developed from CyOTE Precursor Analysis Report data. The Markov model is available as a FastAPI endpoint for seamless integration. Enrichment with Adversary Emulation Capabilities Data: Integrates adversary emulation capabilities data using MITRE Caldera for OT adversary abilities UUIDs. Export of Output Files: Provides options to export the enriched attack chain in JSON or CSV formats. Routing of Output to Other Applications: Facilitates routing of output to various platforms and applications, including: Threat Intelligence Platforms COREII Scout for Threat Intelligence Analysis COREII Modeling and Simulation for Adversary Emulation Technical Description The ACE Application is an advanced cybersecurity tool designed to provide detailed threat analysis and sequence generation. It is built on a robust architecture that integrates natural language processing, machine learning, and historical data modeling. Key Components: Data Ingestion Module: Handles the input of threat reports and data from various formats, ensuring flexibility in data sources. Classification Engine: Employs DeBERTa-based classifiers hosted as FastAPI microservices to analyze and classify threat report events in accordance with the MITRE ATT&CK framework for ICS. Sequence Generator: Orders the classified events into a logical attack chain, providing clear insight into the sequence of tactics and techniques used in the threat. Enrichment Engine: Integrates historical data and adversary emulation capabilities to enhance the attack chain with valuable context and additional details. The historical data enrichment is powered by a Markov model, which is available as a FastAPI endpoint. Export and Routing Module: Facilitates the export of the enriched attack chain in multiple formats and routes the output to designated applications for further analysis or emulation.

Paul, Tony [Idaho National Laboratory (INL), Idaho↗

An Efficient Storage-Driven Machine Learning Model for Performance in the Era of Multimodal Scientific Data

Scientific workflows are increasingly relying on machine learning (ML), simulation, and hybrid techniques to predict, understand, and optimize the behavior of complex experiments. High-performance computing has greatly improved researchers’ ability to acquire diverse data modalities in these workflows. Recent studies suggest that the performance of machine learning models can be improved by integrating data from various sources. Unfortunately, these workloads pose unprecedent pressure on the network storage to meet the demands associated with accessing these multimodal data. To mitigate the impact of intensive IO, we propose a solution that utilizes a multi-tier High-Performance Computing (HPC) distributed storage and data processing framework, placing computation where the data resides for better performance. By adopting this project, the scientific community will gain new opportunities to explore multimodal storage-driven possibilities, integrating multiple scientific data sources with advanced streaming frameworks. Additionally, our framework effectively utilizes computing resources and bridges the gaps identified by HPC experts. Our proposed approach tackles scalability and persistence challenges by leveraging native persistency, which has posed difficulties in traditional approaches. Furthermore, we seek to enhance fault-tolerance and load-balance of computations by leveraging real-time streaming in diverse scientific computing environments, thereby propelling advanced scientific computing research into the next generation.

97 MATHEMATICS AND COMPUTING↗

Laue-DIALS: Open-source software for polychromatic x-ray diffraction data

Most x-ray sources are inherently polychromatic. Polychromatic (“pink”) x-rays provide an efficient way to conduct diffraction experiments as many more photons can be used and large regions of reciprocal space can be probed without sample rotation during exposure—ideal conditions for time-resolved applications. Analysis of such data is complicated, however, causing most x-ray facilities to discard >99% of x-ray photons to obtain monochromatic data. Key challenges in analyzing polychromatic diffraction data include lattice searching, indexing and wavelength assignment, correction of measured intensities for wavelength-dependent effects, and deconvolution of harmonics. We recently described an algorithm, Careless, that can perform harmonic deconvolution and correct measured intensities for variation in wavelength when presented with integrated diffraction intensities and assigned wavelengths. Here, we present Laue-DIALS, an open-source software pipeline that indexes and integrates polychromatic diffraction data. Laue-DIALS is based on the dxtbx toolbox, which supports the DIALS software commonly used to process monochromatic data. As such, Laue-DIALS provides many of the same advantages: an open-source, modular, and extensible architecture, providing a robust basis for future development. We present benchmark results showing that Laue-DIALS, together with Careless, provides a suitable approach to the analysis of polychromatic diffraction data, including for time-resolved applications.

97 MATHEMATICS AND COMPUTING↗

The Role of Data Filtering in Open Source Software Ranking and Selection

Faced with more than 100M open source projects, a more manageable small subset is needed for most empirical investigations. More than half of the research papers in leading venues investigated filtering projects by some measure of popularity with explicit or implicit arguments that unpopular projects are not of interest, may not even represent "real" software projects, or that less popular projects are not worthy of study. However, such filtering may have enormous effects on the results of the studies if and precisely because the sought-out response or prediction is in any way related to the filtering criteria.This paper exemplifies the impact of this common practice on research outcomes, specifically how filtering of software projects on GitHub based on inherent characteristics affects the assessment of their popularity. Using a dataset of over 100,000 repositories, we used multiple regression to model the number of stars -a commonly used proxy for popularity- based on factors such as the number of commits, the duration of the project, the number of authors and the number of core developers. Our control model included the entire dataset, while a second filtered model considered only projects with ten or more authors. The results indicated that while certain characteristics of the repository consistently predict popularity, the filtering process significantly alters the relationships between these characteristics and the response. We found that the number of commits exhibited a positive correlation with popularity in the control sample but showed a negative correlation in the filtered sample. These findings highlight the potential biases introduced by data filtering and emphasize the need for careful sample selection in empirical research of mining software repositories. We recommend that empirical work should either analyze complete datasets such as World of Code, or employ stratified random sampling from a complete dataset to ensure that filtering is not biasing the results.

Malviya Thakur, Addi↗

Thermal Tolerance Metrics for Freshwater Fish, CONUS, Version 1

This dataset is a compilation of 13 thermal response metrics for 834 freshwater fish species across the conterminous United States (CONUS). The data were extracted from six published sources, many of which are compilations of data from other sources. The data were harmonized for comparison, and additional variables were added to summarize the metrics. The dataset is presented as a spreadsheet containing 17 sheets. The first sheet (datasets) describes the data sources. Other sheets describe the source and compilation variables in detail.

13 HYDRO ENERGY↗

A Search for Neutrino Point-source Populations in 7 yr of IceCube Data with Neutrino-count Statistics

The presence of a population of point sources in a data set modifies the underlying neutrino-count statistics from the Poisson distribution. This deviation can be exactly quantified using the non-Poissonian template fitting technique, and in this work we present the first application of this approach to the IceCube high-energy neutrino data set. Using this method, we search in 7 yr of IceCube data for point-source populations correlated with the disk of the Milky Way, the Fermi bubbles, the Schlegel, Finkbeiner, and Davis dust map, or with the isotropic extragalactic sky. No evidence for such a population is found in the data using this technique, and in the absence of a signal, we establish constraints on population models with source-count distribution functions that can be described by a power law with a single break. Here, the derived limits can be interpreted in the context of many possible source classes. In order to enhance the flexibility of the results, we publish the full posterior from our analysis, which can be used to establish limits on specific population models that would contribute to the observed IceCube neutrino flux.

79 ASTRONOMY AND ASTROPHYSICS↗

Subaru Hyper Suprime-Cam Survey of Cygnus OB2 Complex – I. Introduction, photometry, and source catalogue

ABSTRACT Low-mass star formation inside massive clusters is crucial to understand the effect of cluster environment on processes like circumstellar disc evolution, planet, and brown dwarf formation. The young massive association of Cygnus OB2, with a strong feedback from massive stars, is an ideal target to study the effect of extreme environmental conditions on its extensive low-mass population. We aim to perform deep multiwavelength studies to understand the role of stellar feedback on the IMF, brown dwarf fraction and circumstellar disc properties in the region. We introduce here, the deepest and widest optical photometry of 1.5○ diameter region centred at Cygnus OB2 in r2, i2, z, and Y-filters, using Subaru Hyper Suprime-Cam (HSC). This work presents the data reduction, source catalogue generation, data quality checks, and preliminary results about the pre-main sequence sources. We obtain 713 529 sources in total, with detection down to ∼28, 27, 25.5, and 24.5 mag in r2, i2, z, and Y-band, respectively, which is ∼3 – 5 mag deeper than the existing Pan-STARRS and GTC/OSIRIS photometry. We confirm the presence of a distinct pre-main sequence branch by statistical field subtraction of the central 18 arcmin region. We find the median age of the region as ∼5 ± 2 Myr with an average disc fraction of ∼9 per cent. At this age, combined with A $_V\, \sim$ 6 – 8 mag, we detect sources down to a mass range of ∼0.01–0.17 M⊙. The deep HSC catalogue will serve as the groundwork for further studies on this prominent active young cluster.

Gupta, Saumya (ORCID:0000000161843958)↗

Hawthorne Nevada Deep Direct-Use Feasibility Study - Data Used for Geothermal Resource Conceptual Modeling and Power Capacity Estimates

This data submission includes several data components that were used to develop a conceptual model and power capacity-estimates of two low-temperature geothermal resources that define geothermal prospect A at Hawthorne, Nevada. Data are sourced from a combination of legacy publicly-available data and more recent data acquisition conducted by the US Navy Geothermal Program Office (2008-2013) and the Great Basin Center for Geothermal Energy at the University of Nevada, Reno (2008-2010). Data sets include compiled fluid geochemistry data, down-hole temperature logs for wells in the vicinity of prospect A, 2 meter temperature survey data, temperature-spinner logs acquired in well HWAAD-2A, fracture picks from image log data acquired in wells HWAAD-2 and HWAAD-3, and X-Ray Diffraction (XRD) analyses on cuttings from wells HWAAD-2A and HWAAD-3. These data have been reviewed for errors and inconsistencies, but it is possible that few errors could still remain. The resource conceptual model and power capacity estimates are included in the final report to the US Department of Energy, and are presented in a manuscript by Ayling and Hinz. A link to the manuscript published in Geothermics is linked below in this submission.

15 GEOTHERMAL ENERGY↗

Freight Analysis Framework Version 5 (FAF5) Base Year 2017 Data Development Technical Report

The Freight Analysis Framework (FAF) integrates data from a variety of sources to create a comprehensive national picture of freight movements among states and major metropolitan areas by all modes of transportation. The latest of this data series is FAF5, which is the fifth generation FAF and is benchmarked on Commodity Flow Survey (CFS) 2017. Except for FAF1 that provided estimates for truck, rail, and water tonnage for calendar year 1998, later generations of FAF (FAF2 through FAF5) were built based on their benchmark year CFS data, for 2002, 2007, 2012, and 2017 respectively. The FAF is produced under a partnership between Bureau of Transportation Statistics (BTS) and Federal Highway Administration (FHWA). As a major data product of the FAF program, the FAF regional database provides a national picture of freight flows to, from, and within the United States (among regions and states), by commodity and mode for the base year, as well as for forecasts up to 30 years into the future in a 5-year interval. Additional FAF data products also include FAF network flows database, where truck movements are routed onto the national highway network, estimates of annual projections, and synchronized historical data series. This report is a technical document prepared to describe the data sources and methodologies applied in the process of building the FAF5 base-year 2017 regional database, released as FAF5.0 in February 2021. This report offers a description of the diverse data sources and modeling methods used in constructing the base year FAF5 regional database. The FAF5 base-year database is used as the base for development of forecasts and for assignment of truck flows on highway network. Similarly, the FAF5 base-year database will be used as the base to generate FAF5 annual estimates. In addition to this report, users are encouraged to refer to the FAF5 User’s Guide, which provides basic information of the data, including definitions of the data attributes, information on how to access the data and tool, as well as detailed data dictionary and code tables.

42 ENGINEERING↗

Equation-based and data-driven modeling: Open-source software current state and future directions

Here, a review of current trends in scientific computing reveals a broad shift to open-source and higher-level programming languages such as Python and growing career opportunities over the next decade. Open-source modeling tools accelerate innovation in equation-based and data-driven applications. Significant resources have been deployed to develop data-driven tools (PyTorch, TensorFlow, Scikit-learn) from tech companies that rely on machine learning services to meet business needs while keeping the foundational tools open. Open-source equation-based tools such as Pyomo, CasADi, Gekko, and JuMP are also gaining momentum according to user community and development pace metrics. Integration of data-driven and principles-based tools is emerging. New compute hardware, productivity software, and training resources have the potential to radically accelerate progress. However, long-term support mechanisms are still necessary to sustain the momentum and maintenance of critical foundational packages.

97 MATHEMATICS AND COMPUTING↗

Lead isotopic compositions of airborne particulates collected from snow and from dry deposition in the East River Watershed, Colorado and at the Southern Sierra Critical Zone Observatory, California

The data set consists of lead (Pb) isotopic ratios (206Pb/207Pb and 208Pb/207Pb, as well as 206Pb/204Pb, 207Pb/204Pb, 208Pb/204Pb) for particulate samples collected from snow and dry deposition. This was part of a study (O’Day et al. 2020) that was looking at the speciation of phosphorous in splits of these samples to evaluate the flux and bio-availability of phosphorous to the mountain sites in the study. The Pb isotopic data allow sourcing of long-range transported Pb, specifically the fraction of Pb sourced in Asia, and so a comparison of the relative distances of PM sources between the high and low elevation sites. Samples were collected at the East River Watershed, CO (dry deposition collected over Sept.-Oct. 2016, show sampling winter 2017) and at the Southern Sierra Critical Zone Observatory (dry deposition collected over Sept-Nov 2016, July-Sept 2017, June-July 2017). The data includes Pb concentration data, as well as calculated fraction of the Pb sourced in Asia. This data set was published in part in O’Day et al. (2020) Phosphorus Speciation in Atmospherically Deposited Particulate Matter and Implications for Terrestrial Ecosystem Productivity. ES&T, https://dx.doi.org/10.1021/acs.est.9b06150

54 ENVIRONMENTAL SCIENCES↗