SEARCH · Engineering Papers

Results for “FILES”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 901 records · Page 50

All sources NASA NTRS DOE OSTI

NASA NTRS1987

Manual for Getdata Version 3.1: a FORTRAN Utility Program for Time History Data

This report documents version 3.1 of the GetData computer program. GetData is a utility program for manipulating files of time history data, i.e., data giving the values of parameters as functions of time. The most fundamental capability of GetData is extracting selected signals and time segments from an input file and writing the selected data to an output file. Other capabilities include converting file formats, merging data from several input files, time skewing, interpolating to common output times, and generating calculated output signals as functions of the input signals. This report also documents the interface standards for the subroutines used by GetData to read and write the time history files. All interface to the data files is through these subroutines, keeping the main body of GetData independent of the precise details of the file formats. Different file formats can be supported by changes restricted to these subroutines. Other computer programs conforming to the interface standards can call the same subroutines to read and write files in compatible formats.

Maine, Richard E.↗

NASA NTRS1998

A Geometry Based Infra-Structure for Computational Analysis and Design

The computational steps traditionally taken for most engineering analysis suites (computational fluid dynamics (CFD), structural analysis, heat transfer and etc.) are: (1) Surface Generation -- usually by employing a Computer Assisted Design (CAD) system; (2) Grid Generation -- preparing the volume for the simulation; (3) Flow Solver -- producing the results at the specified operational point; (4) Post-processing Visualization -- interactively attempting to understand the results. For structural analysis, integrated systems can be obtained from a number of commercial vendors. These vendors couple directly to a number of CAD systems and are executed from within the CAD Graphical User Interface (GUI). It should be noted that the structural analysis problem is more tractable than CFD; there are fewer mesh topologies used and the grids are not as fine (this problem space does not have the length scaling issues of fluids). For CFD, these steps have worked well in the past for simple steady-state simulations at the expense of much user interaction. The data was transmitted between phases via files. In most cases, the output from a CAD system could go to Initial Graphics Exchange Specification (IGES) or Standard Exchange Program (STEP) files. The output from Grid Generators and Solvers do not really have standards though there are a couple of file formats that can be used for a subset of the gridding (i.e. PLOT3D data formats). The user would have to patch up the data or translate from one format to another to move to the next step. Sometimes this could take days. Specifically the problems with this procedure are:(1) File based -- Information flows from one step to the next via data files with formats specified for that procedure. File standards, when they exist, are wholly inadequate. For example, geometry from CAD systems (transmitted via IGES files) is defined as disjoint surfaces and curves (as well as masses of other information of no interest for the Grid Generator). This is particularly onerous for modern CAD systems based on solid modeling. The part was a proper solid and in the translation to IGES has lost this important characteristic. STEP is another standard for CAD data that exists and supports the concept of a solid. The problem with STEP is that a solid modeling geometry kernel is required to query and manipulate the data within this type of file. (2) 'Good' Geometry. A bottleneck in getting results from a solver is the construction of proper geometry to be fed to the grid generator. With 'good' geometry a grid can be constructed in tens of minutes (even with a complex configuration) using unstructured techniques. Adroit multi-block methods are not far behind. This means that a million node steady-state solution can be computed on the order of hours (using current high performance computers) starting from this 'good' geometry. Unfortunately, the geometry usually transmitted from the CAD system is not 'good' in the grid generator sense. The grid generator needs smooth closed solid geometry. It can take a week (or more) of interaction with the CAD output (sometimes by hand) before the process can begin. One way Communication. (3) One-way Communication -- All information travels on from one phase to the next. This makes procedures like node adaptation difficult when attempting to add or move nodes that sit on bounding surfaces (when the actual surface data has been lost after the grid generation phase). Until this process can be automated, more complex problems such as multi-disciplinary analysis or using the above procedure for design becomes prohibitive. There is also no way to easily deal with this system in a modular manner. One can only replace the grid generator, for example, if the software reads and writes the same files. Instead of the serial approach to analysis as described above, CAPRI takes a geometry centric approach. This makes the actual geometry (not a discretized version) accessible to all phases of the analysis. The connection to the geometry is made through an Application Programming Interface (API) and NOT a file system. This API isolates the top-level applications (grid generators, solvers and visualization components) from the geometry engine. Also this allows the replacement of one geometry kernel with another, without effecting these top-level applications. For example, if UniGraphics is used as the CAD package then Parasolid (UG's own geometry engine) can be used for all geometric queries so that no solid geometry information is lost in a translation. This is much better than STEP because when the data is queried, the same software is executed as used in the CAD system. Therefore, one analyzes the exact part that is in the CAD system. CAPRI uses the same idea as the commercial structural analysis codes but does not specify control. Software components of the CAD system are used, but the analysis suite, not the CAD operator, specifies the control of the software session. This also means that the license issues (may be) minimized and individuals need not have to know how to operate a CAD system in order to run the suite.

Haimes, Robert↗

NASA NTRS2011

Framework for Integrating Science Data Processing Algorithms Into Process Control Systems

A software framework called PCS Task Wrapper is responsible for standardizing the setup, process initiation, execution, and file management tasks surrounding the execution of science data algorithms, which are referred to by NASA as Product Generation Executives (PGEs). PGEs codify a scientific algorithm, some step in the overall scientific process involved in a mission science workflow. The PCS Task Wrapper provides a stable operating environment to the underlying PGE during its execution lifecycle. If the PGE requires a file, or metadata regarding the file, the PCS Task Wrapper is responsible for delivering that information to the PGE in a manner that meets its requirements. If the PGE requires knowledge of upstream or downstream PGEs in a sequence of executions, that information is also made available. Finally, if information regarding disk space, or node information such as CPU availability, etc., is required, the PCS Task Wrapper provides this information to the underlying PGE. After this information is collected, the PGE is executed, and its output Product file and Metadata generation is managed via the PCS Task Wrapper framework. The innovation is responsible for marshalling output Products and Metadata back to a PCS File Management component for use in downstream data processing and pedigree. In support of this, the PCS Task Wrapper leverages the PCS Crawler Framework to ingest (during pipeline processing) the output Product files and Metadata produced by the PGE. The architectural components of the PCS Task Wrapper framework include PGE Task Instance, PGE Config File Builder, Config File Property Adder, Science PGE Config File Writer, and PCS Met file Writer. This innovative framework is really the unifying bridge between the execution of a step in the overall processing pipeline, and the available PCS component services as well as the information that they collectively manage.

Mattmann, Chris A.↗

NASA NTRS2013

ISTP CDF Skeleton Editor

Basic Common Data Format (CDF) tools (e.g., cdfedit) provide no specific support for creating International Solar-Terrestrial Physics/Space Physics Data Facility (ISTP/SPDF) standard files. While it is possible for someone who is familiar with the ISTP/SPDF metadata guidelines to create compliant files using just the basic tools, the process is error-prone and unreasonable for someone without ISTP/SPDF expertise. The key problem is the lack of a tool with specific support for creating files that comply with the ISTP/SPDF guidelines. There are basic CDF tools such as cdfedit and skeletoncdf for creating CDF files, but these have no specific support for creating ISTP/ SPDF compliant files. The SPDF ISTP CDF skeleton editor is a cross-platform, Java-based GUI editor program that allows someone with only a basic understanding of the ISTP/SPDF guidelines to easily create compliant files. The editor is a simple graphical user interface (GUI) application for creating and editing ISTP/SPDF guideline-compliant skeleton CDF files. The SPDF ISTP CDF skeleton editor consists of the following components: A swing-based Java GUI program, JavaHelp-based manual/ tutorial, Image/Icon files, and HTML Web page for distribution. The editor is available as a traditional Java desktop application as well as a Java Network Launching Protocol (JNLP) application. Once started, it functions like a typical Java GUI file editor application for creating/editing application-unique files.

Chimiak, Reine↗

DOE OSTI2022

Organic matter concentration and composition of experimentally burned open air and muffle furnace vegetation chars across differing burn severity and feedstock types from Pacific Northwest, USA (v4).

This dataset represents results from an experimental study designed to compare how the chemical composition of organic matter changes across different burn conditions and feedstock materials. The dataset provides both solid and dissolved phase bulk concentration and organic matter characterization data from experimentally generated chars. Chars were created in a closed muffle furnace or on an open burn table from four different feedstock species representing vegetation commonly impacted by fire regimes across the Pacific Northwest, USA. This data can be used to compare how different burn conditions may influence resultant organic matter chemistry and help further our understanding of potential biogeochemical impacts on river corridors post-fire. This dataset is comprised of one data package readme, one data dictionary (dd), one file level metadata (flmd), fourteen burn table videos, burn table video metadata and three folders containing (A) data; (B) metadata and protocols; and (C) photos. The folder names and the file name of the data package readme include a version number which will be updated with future iterations of this data package. The data folder includes (1) solid carbon and solid nitrogen; (2) dissolved organic carbon (DOC, measured as non-purgeable organic carbon, NPOC) and total dissolved nitrogen (TN); (3) pH; (4) thermocouple time series temperature; (5) methods codes; (6) installation methods; (7) excitation emissions matrix (EEM) methods information; (8) a folder of excitation emissions matrix (EEM) fluorescence and absorbance spectra in dissolved organic matter and EEMs processing instructions; (9) solid state carbon-13 and solution state phosphorus nuclear magnetic resonance (13-C NMR and 31-P NMR) data and methods; (10) benzene polycarboxylic acid (BPCA) concentration and stable isotope data; (11) FTICR-MS methods; (12) Inductively coupled plasma (ICP) data for total calcium, magnesium, iron, aluminum, potassium, phosphorus, sodium, and sulfur along with sodium hydroxide-ethylenediaminetetraacetic acid (sodium hydroxide-EDTA) extractable calcium, magnesium, iron, aluminum, potassium, phosphorus, and sulfur; (13) a folder of phosphorus, carbon, and nitrogen X-ray absorption near edge structure (P-XANES, N-XANES, C-XANES) data for samples and standards; (14) P-XANES, N-XANES, C-XANES methods; (15) molybdate reactive phosphorus; and (16) folder of high resolution characterization of organic matter via 21 Tesla Fourier transform ion cyclotron resonance mass spectrometry (FTICR-MS) generated through the Environmental Molecular Sciences Laboratory (EMSL; https://www.pnnl.gov/environmental-molecular-sciences-laboratory). The FTICR folder contains .txt data files and a subfolder containing instruments for using Formularity (https://omics.pnl.gov/software/formularity) and an R script to process the data based on the user's specific needs. The metadata and protocols folder includes (1) international geo-sample number (IGSN) mapping file (2) burn and laboratory metadata; (3) burn protocol; (4) laboratory protocol; (5) vegetation collection metadata; and (6) vegetation collection protocol. The folder contains photos of the solid chars. All files are .csv, .txt, .pdf, .jpg, .jpeg, .R, .ref, or .mp4. The data package was originally published October 2022 (v1). It was updated April 2023 (v2; new data files), September 2023 (v3; new and corrected data files), and September 2024 (v4; new and added/updated files). Metadata files were also updated to reflect these changes. See the change history section in the readme for more details.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Leaf phenology data at The Morton Arboretum Forestry Plots 2019-2023

We are collecting long-term leaf phenology data at The Morton Arboretum to determine seasonal patterns of leaf production in trees. This data on leaf phenology will be integrated with other ongoing data streams to create a connection between above- and below-ground tree processes. This data package contains raw and smooth outputs from phenology data, as well as extracted phenophase dates (i.e., start, peak, and end of season): the raw and smooth outputs from the PhenoCam GUI can be found in the "leafRaw.csv" and "leafSmooth.csv" files, respectively, and the extracted phenophase dates can be found in the "leafPhenophaseDates2019-2023.csv" file. Extracted phenophase dates for evergreen species in 2023 are currently unavailable, and the files will be updated once they are extracted. Additional information on units and other file-level metadata can be found within each data file's respective data dictionary, and metadata for each of the 23 surveyed plots can be found within the "Location_metadata.csv" file. While the "leafRaw.csv" and the "leafSmooth.csv" files contain all data for all species, the "leafPhenophaseDates2019-2023.csv" file currently excludes the dates for evergreen species in 2023. Another version of the file will be added as those dates are extracted.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Metagenome-assembled genomes from Wind River Basin floodplain sediments Riverton, Wyoming site (June to October 2019)

Microorganisms play a key role in cycling nutrients and contaminants in the terrestrial environment depending on their genetic potential. Here we present metagenome-assembled genomes (MAGs) for the bacterial and archaeal community in floodplain sediment samples taken at three time points from June 12, 2019 to October 23,2019 at a location (PTT1) close to DOE Legacy Management well 855 at the Riverton, Wyoming floodplain site in the Wind River Basin (WRB). The groundwater at this site exhibits persistent U, Mo, and sulfate plumes and is one of the field sites in focus for the SLAC Groundwater Quality SFA program. Sediment samples were collected from 60 to 180 cm below surface every 30cm for microbial analyses through metagenomic sequencing. 15 metagenomes were sequenced through JGI and can be found under Gold sequencing project: Gs0131241. Metagenomes were assembled, binned, and refined using metawrap to generate MAGs (>50% complete and < 10% contamination based on checkM scores). This dataset includes a zip file of 780 MAG fasta files and a csv file with quality, taxonomic classification (GTDB RS220), and metagenome accessions for MAGs. This dataset also includes a file-level metadata (flmd.csv) file that lists each file contained in the dataset with associated metadata and a data dictionary (dd.csv) file that contains column/row headers used throughout the files along with a definition, units, and data type. A sample metadata file (samples.csv) that contains site information has also been included.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Montane Conifer, Aspen, Meadow, and Sagebrush Metagenome Resolved Genomes and Traits in East River Watershed, Colorado, USA

Climate change is driving vegetation shifts in mountain watersheds, with unknown impacts on biogeochemical cycles. We hypothesize that these shifts will reshape soil microbiomes and associated biogeochemical processes. As a part of Lawrence Berkeley National Laboratory (LBNL) Watershed Science Focus Area (SFA), we assessed microbiome and microbial functional trait differences between soils under conifer, aspen, forby meadows, and sagebrush across the East River Watershed, CO, controlling for elevation and aspect.Here we present metagenome assembled genomes (MAGs) for the bacterial and archaeal communities from soils 0-20cm in depth across three locations in the watershed—Headwaters, Upper Reaches, and Lower Reaches from August 3-11th 2016. Each location was further subdivided into two blocks, with one block on a west facing aspect, and two on the east aspect of the valley. Within blocks, two samples per vegetation type were taken (one at each depth). This resulted in 66 samples, which were sequenced at JGI and can be found under the Joint Genome Institute (JGI) Genomes Online Database (GOLD) sequencing project Gs0118068. Metagenomes were assembled through an inhouse pipeline (see methods), binned using four autobinners (concoct, maxbin2, metabat2, and vamb) and consolidated using dastool. The consolidated bins from all metagenomes were pooled, filtered by completeness (>75%) and contamination (<25%), and dereplicated at 95% ANI using drep. The dataset includes a zip file of 687 genomes (Vegtype_MAGS.zip), the accession numbers for the underlying metagenomes, a csv file with MAG quality metrics and taxonomy from Genome Taxonomy Database (GTDB) and National Center for Biotechnology Information (NCBI) taxonomic representative genome proteins (EastRiver_Vegtype_drep_genome_info.csv), and a file containing MAG quality metrics and taxonomy (gtdb_drep_bin_taxonomy.csv). The dataset additionally includes a sample metadata file (EastRiver_Vegtype_sample_metadata.csv), a metadata file used to register associated samples with IGSNs (International Generic Sample Numbers) (samples.csv), a Google KML file for the sampled locations (sample_collection_sites.kml), a location metadata file (locations.csv), a file-level metadata file (flmd.csv), and a data dictionary (dd.csv) file.This work was supported by the Watershed Function Science Focus Area at Lawrence Berkeley National Laboratory funded by the US Department of Energy, Office of Science, Biological and Environmental Research under Contract No. DE-AC02-05CH11231.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Metagenome-assembled genomes from soil samples in control and warming plots in Blodgett Forest, CA (2014-2021)

The pathways of carbon transport and loss through and from soils—soil organic matter (SOM) depolymerization to dissolved organic carbon and mineralization to carbon dioxide (CO2)—are fundamentally driven by microbial activity, which is strongly regulated by environmental conditions. As part of LBNL (Lawrence Berkeley National Laboratory) TES (Terrestrial Ecosystem Science) Belowground Biogeochemistry Science Focus Area (SFA), we have established a novel whole-soil long-term warming experiment at the University of California (UC) Blodgett Forest Research Station (Sierra Nevada) in 2014, where we study the role of biogeochemical, microbial and geochemical process interactions in SOM decomposition and stabilization.Here, we present metagenome-assembled genomes (MAGs) for the bacterial and archaeal community from soil depth profiles collected from 2014 to 2021 from three paired control and warming plots. We collected soil samples across a range of depth profiles (spanning surface to 90 cm deep) from three paired control and warming plots from a temperate mixed forest in Northern California. Each paired plot had been subjected to experimental warming since June 2014 to simulate a predicted climate change scenario for northern California. 101 soil metagenomes were sequenced at JGI (Joint Genome Institute) and UCSF (University of California San Francisco) Center for Advanced Technology and can be found under the JGI (Joint Genome Institute) GOLD (Genomes Online Database) Sequencing project Gs0151586 and NCBI (National Center for Biotechnology Information) Projects PRJNA1225762 and PRJEB39497. Metagenomes were assembled using JGI (Joint Genome Institute) Metagenome Workflow (10.1128/mSystems.00804-20). For each metagenome, the assembled contigs were binned into genomes using 3 binning algorithms (cocacola, metabat, and maxbin) and the resulting bins were consolidated using dastool. The consolidated bins from all metagenomes were pooled, filtered by completeness (>50%) and contamination (<25%), and dereplicated at 99% ANI (average nucleotide identity) using dRep (https://github.com/MrOlm/drep).The dataset includes a zip file of 2321 MAG (Metagenome Assembled Genome) fasta files, the accession numbers for the underlying metagenomes, and a csv file with MAG (Metagenome Assembled Genome) quality metrics and taxonomic classification (GTDB -Genome Taxonomy Database-RS220). This dataset also includes a file-level metadata (flmd.csv) file that lists each file contained in the dataset with associated metadata and a data dictionary (dd.csv) file that contains column/row headers used throughout the files along with a definition, units, and data type. A sample metadata file (samples.csv) that contains site information has also been included.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2026

Post-fire time series of sensor and geochemistry sample data from surface water, groundwater, precipitation, soil, and vegetation across Oak Creek watershed, Washington

This dataset supports a broader study examining wildfire impacts on hydrologic connectivity across 5 sites within the Oak Creek watershed and the resulting biogeochemical impacts. Stream sites were selected using the Advanced Terrestrial Simulator (ATS) hydrologic model to identify locations with varying groundwater contributions and hydrologic responses across different burn severity scenarios. The Retreat Fire burned from July 23 to August 2 in 2024, affecting the five study sites at varying burn severities. Each site is equipped with YSI EXO2 sondes logging sub-hourly throughout the year, and grab samples are collected approximately every six weeks. YSI sondes are used to measure temporally resolved proxies for groundwater inputs (specific conductivity) and organic matter (fluorescent dissolved organic matter; fDOM) along with basic water quality and depth. Grab samples of surface water, groundwater, and precipitation are analyzed for water stable isotopes and conductivity to understand endmembers for hydrologic mixing Grab samples of surface water, groundwater, soil water, and litter/vegetation/soil leachates are analyzed for organic matter composition measured by Fourier-Transform Ion Cyclotron Resonance Mass Spectrometry (FTICR-MS) to understand organic matter dynamics. Game camera photos are provided in a separate data package available at https://data.ess-dive.lbl.gov/datasets/doi:10.15485/3018598. Future versions of this dataset will include time series data from YSI EXO2 sondes (fDOM, dissolved oxygen, temperature, depth, specific conductance, turbidity, pH), BaroTROLL sensors (air temperature and barometric pressure), rain gauges (precipitation), and data from the soil and vegetation samples. Because this study is ongoing, this data package will be updated regularly to include newly collected data and the additional data types. For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. In addition to a readme, this data package also includes a file-level metadata (FLMD) file that describes each file and a data dictionary (DD) that describes all column/row headers and variable definitions. This dataset is comprised of (1) a folder of field photos; (2) a folder of surface water sample data; (3) a folder of raw Fourier transform ion cyclotron resonance mass spectrometry (FTICR-MS) data; (4) a data checks report; (5) file-level metadata; (6) data dictionary; (7) field metadata; (8) readme; (9) international generic sample number (IGSN) mapping file; and (10) field protocols. The sample data subfolder contains (1) dissolved organic carbon (DOC, measured as non-purgeable organic carbon, NPOC) data and averages; (2) total dissolved nitrogen data and averages; (3) stable water isotopes and averages; (4) methods codes; (5) FTICR-MS methods; and (15) a subfolder of 9.4 Tesla (9.4T) FTICR-MS data. This folder contains the processed data and three subfolders, one containing the .xml files, one containing the water CoreMS output files, and the other containing instructions and scripts for processing the files in CoreMS (https://github.com/EMSL-Computing/CoreMS). All files are .csv, .pdf, .R, .xml, .d, .html, .Rmd, .py, .cal, .json, .jpg, .jpeg, .png, .mov, or .mp4.

Biogeochemistry↗

DOE OSTI2026

Dated soil C–N–P profiles, water quality, and chamber fluxes across Ohio and Michigan wetlands (2024–2025)

This dataset includes dated soil core chemistry (bulk density, phosphorus, nitrogen and carbon concentrations), water quality, and chamber flux measurements collected from wetlands in the Midwest United States—12 sites in Ohio, one site in Indiana, one site in Michigan—collected in the spring or summer of 2024 or 2025, all in (.csv) format. These data were generated to examine how wetland restoration, management activities, and time since restoration affect biogeochemical processes, carbon sequestration, nutrient accumulation, water quality, and greenhouse gas emissions. Specifically, these data aim to investigate how restored wetlands differ from natural wetlands in terms of carbon, nitrogen, phosphorus dynamics, as well as carbon dioxide and methane fluxes. Also included are surface and porewater quality parameters and chamber flux measurements across these different wetlands. Sampling was conducted at various sites representing a range of restoration stages, from about 4 years post-restoration up to 105 years post-restoration, and also includes a natural wetland used as a reference in Michigan. These data can be used to determine carbon sequestration rates, nutrient cycling, and to enhance our understanding of biogeochemical responses to wetland restoration in temperate ecosystems. This data package contains (1) a csv file (Water_Quality.csv) containing water quality data (dissolved organic carbon, total dissolved nitrogen, and temperature) organized by location; (2) a csv file (Soil_C_N_P_Seq.csv) containing carbon, nitrogen, and phosphorus concentrations at each soil level and time of each soil level, as well as their sequestration rates; (3) a csv file (CH4_CO2_Flux.csv) including methane and carbon dioxide fluxes that were measured with a chamber; (4) a file-level metadata (FLMD.csv) file that lists each file contained in the dataset with associated metadata; (5) a data dictionary (DD.csv) file that contains terms/column headers used throughout the files along with a definition, units, and data type; and (6) a locations metadata file (Location_metadata.csv).

Earth Science > Atmosphere > Atmospheric Chemistry↗

DOE OSTI2026

WHONDRS Surface Water Geochemistry and Organic Matter Characterization Data from Streams Distributed across Latin America

This dataset supports a broader study examining global transferability of stream biogeochemistry and was generated in collaboration with the MicroSudAqua (µSudAqua) network (https://microsudaqua.netlify.app/en/). The dataset provides surface water geochemistry (dissolved organic carbon, total dissolved nitrogen, cations) and organic matter characterization (FTICR-MS) from streams in Argentina, Brazil, Chile, and Colombia. Samples were collected across stream orders (1st to 6th order) within five basins. Related data were collected and will be published separately in collaboration with the µSudAqua network. For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. In addition to this readme, this data package also includes a file-level metadata (FLMD) file that describes each file and a data dictionary (DD) that describes all column/row headers and variable definitions. This dataset is comprised of (1) a folder of field photos; (2) a folder of surface water sample data, (3) a folder of raw Fourier transform ion cyclotron resonance mass spectrometry (FTICR-MS) data; (4) file-level metadata; (5) data dictionary; (6) field metadata; (7) readme; (8) international generic sample number (IGSN) mapping file; and (9) field protocol. The sample data subfolder contains (1) dissolved organic carbon (DOC, measured as non-purgeable organic carbon, NPOC) data and averages; (2) total dissolved nitrogen data and averages; (3) anions and averages; (4) methods codes; (5) FTICR-MS methods; and (15) a subfolder of 9.4 Tesla (9.4T) FTICR-MS data. This folder contains the processed data and three subfolders, one containing the .xml files, one containing the water CoreMS output files, and the other containing instructions and scripts for processing the files in CoreMS (https://github.com/EMSL-Computing/CoreMS). All files are .csv, .pdf, .R, .xml, .d, .html, .Rmd, .py, .cal, .json, .jpg, .jpeg, .png, .mov, or .mp4.

Anions↗

DOE OSTI2024

Frost Tables, NGEE Areas B, C and D, Utqiagvik (Barrow), Alaska, 2012-2014

This dataset represents spatially intensive thaw depth surveys with individual point measurements spaced ~0.5 m apart. The three ~10x10m grids cover an ice wedge and a portion of its two neighboring polygons. The data contain thaw depth, frost table elevation, ground surface elevation, active layer depth and surface water inundation across three seasons (2012, 2013 and 2014) at Utqiagvik (Barrow) NGEE Areas B, C and D. There are a total of 69 extracted files separated by month, year, and area. The data were originally provided in excel workbooks, these files have been preserved within the zip "Original_Files." The files were transformed for archiving with images pulled out into JPEGs, data preserved as CSV files and some methodological information preserved as TXT files. Some functions, features, and formatting were lost during the transformation. Missing data were defined with blank cells. Additional documentation can be found in files labelled with "information.txt" file endings.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).[2024-07-15] Renamed BEO_Thaw_Depth_Frost_Table_Area_*_*_*.csv, BEO_Thaw_Depth_Frost_Table_Area_*_*_*_Plot_1.JPEG, and BEO_Thaw_Depth_Frost_Table_Area_*_*_*_Plot_2.JPEG names that had E or L to be Early or Late. Renamed BEO_Thaw_Depth_Frost_Table_Area_*_*_*_Plot_1.JPEG, BEO_Thaw_Depth_Frost_Table_Area_*_*_*_Plot_2.JPEG, and BEO_Thaw_Depth_Frost_Table_Area_*_information_Plot_1.JPEG to BEO_Thaw_Depth_Frost_Table_Area_*_*_*_Gridpoints.JPEG, BEO_Thaw_Depth_Frost_Table_Area_*_*_*_3Dgraphic.JPEG, and BEO_Thaw_Depth_Frost_Table_Area_*_information_LiDAR_DEM.JPEG, respectively. Updated BEO_Thaw_Depth_Frost_Table_flmd.csv file to reflect above updates.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2016

Ground Water Levels for NGEE Areas A, B, C and D, Utqiagvik (Barrow), Alaska, 2012-2014

Ice wedge polygonal tundra water levels were measured at a total of 45 locations representing polygon centers and troughs during three summers. Early season water levels, which were still affected by ice and snow, are represented by manual measurements only. Continuous (less than hourly) measurements followed through early fall (~mid-Sep). The data set contains inundation depth (cm), absolute water level and local ground surface elevation (masl). There are a total of 209 files in this dataset. The data were originally provided in Excel workbooks and these files have been preserved within the zip "Original_Files." The workbooks were also transformed and separated by well ID, year, and/or area into 205 extracted files. The Excel files were transformed for archiving with images pulled out into JPEGs, data preserved as CSV files and some methodological information preserved as TXT files. Some functions, features, and formatting were lost during the transformation. Missing data were defined with blank cells. Additional documentation can be found in files labelled with "information.txt" file endings.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

Beyond the Hype: An Evaluation of Commercially Available Machine-Learning-Based Malware Detectors

There is a lack of scientific testing of commercially available malware detectors, especially those that boast accurate classification of never-before-seen (i.e., zero-day) files using machine learning (ML). Consequently, efficacy of malware detectors is opaque, inhibiting end users from making informed decisions and researchers from targeting gaps in current detectors. In this paper, we present a scientific evaluation of four prominent commercial malware detection tools to assist an organization with two primary questions: To what extent do ML-based tools accurately classify previously and never-before-seen files? Is purchasing a network-level malware detector worth the cost? To investigate, we tested each tool against 3,536 total files (2,554 or 72% malicious, 982 or 28% benign) of a variety of file types, including hundreds of malicious zero-days, polyglots, and APT-style files, delivered on multiple protocols. We present statistical results on detection time and accuracy, consider complementary analysis (using multiple tools together), and provide two novel applications of the recent cost-benefit evaluation procedure of Iannacone & Bridges. Although the ML-based tools are more effective at detecting zero-day files and executables, the signature-based tool might still be an overall better option. Both network-based tools provide substantial (simulated) savings when paired with either host tool, yet both show poor detection rates on protocols other than HTTP or SMTP. Our results show that all four tools have near-perfect precision but alarmingly low recall, especially on file types other than executables and office files—37% of malware, including all polyglot files, were undetected. Priorities for researchers and takeaways for end users are given. Code for future use of the cost model is provided.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2012

Amalgam

Amalgam is a system that allows you to generate collections of files that work with conventional tools but take up minimal extra space on storage. If you already are archiving files in a file system and don't want to make multiple copies of whole files in order to ship the data somewhere then this system is for you. Amalgam consists of tools that allow you to create and get information about '.amalgam' files which contain just the metadata associated with a set of files. It also contains a fuse file system that, using amalgam files, will dynamically create traditional tar files. With this setup, you can see multiple orders of magnitude reduction in storage requirements for temporary data collections.

Fox, Kevin↗

DOE OSTI2011

Bulk Data Mover

Bulk Data Mover (BDM) is a high-level data transfer management tool. BDM handles the issue of large variance in file sizes and a big portion of small files by managing the file transfers with optimized transfer queue and concurrency management algorithms. For example, climate simulation data sets are characterized by large volume of files with extreme variance in file sizes. The BDN achieves high performance using a variety of techniques, including multi-threaded concurrent transfer connections, data channel caching, load balancing over multiple transfer servers, and storage i/o pre-fetching. Logging information from the BDM is collected and analyzed to study the effectiveness of the transfer management algorithms. The BDM can accept a request composed of multiple files or an entire directory. The request also contains the target site and directory where the replicated files will reside. If a directory is provided at the source, then the BDM will replicate the structure of the source directory at the target site. The BDM is capable of transferring multiple files concurrently as well as using parallel TCP streams. The optimal level of concurrency or parallel streams depends on the bandwidth capacity of the storage systems at both ends of the transfer as well as achievable bandwidth of the wide-area network. Hardware req.-PC, MAC, Multi-platform & Workstation; Software req.: Compile/version-Java 1.50_x or above; Type of files: source code, executable modules, installation instructions other, user guide; URL: http://sdm.lbl.gov/bdm/

Natarajan, Vijaya↗

DOE OSTI2012

Purge Lock Server

The software provides a simple web api to allow users to request a time window where a file will not be removed from cache. HPSS provides the concept of a "purge lock". When a purge lock is set on a file, the file will not be removed from disk, entering tape only state. A lot of network file protocols assume a file is on disk so it is good to purge lock a file before transferring using one of those protocols. HPSS's purge lock system is very coarse grained though. A file is either purge locked or not. Nothing enforces quotas, timely unlocking of purge locks, or managing the races inherent with multiple users wanting to lock/unlock the same file. The Purge Lock Server lets you, through a simple REST API, specify a list of files to purge lock and an expire time, and the system will ensure things happen properly.

Fox, Kevin↗