Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “big data transfer”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

58 records · Page 4

Characterization of shale using Helium and Argon at high pressures

In order to estimate the shale gas in place and the eventual recovery during shale gas operations, one of the key requirements is to accurately characterize the shale’s petrophysical and transport properties such as porosity, permeability, diffusivity, and storage capacity. Despite the many efforts reported in the technical literature aiming to provide an improved understanding of the complex pore structures and the associated fluid flow in gas shales, complete characterization of organic-rich shale samples still poses a big challenge. Here, we have characterized mass transfer and sorption in shale at different length scales using Helium (He) and Argon (Ar) as probe gases. Thermogravimetric analysis (TGA) with a shale cube of ~1 cm 3 in size and gas expansion experiments with a full-diameter core (3.5” in diameter) were used to measure sorption kinetics/isotherms and mass transfer, respectively. Both samples are from the same depth/location in the Marcellus shale formation. The TGA steady-state technique was initially used to generate excess sorption isotherms for Ar, while dynamic TGA experiments were used to study its sorption kinetics. The TGA experiments demonstrate that Ar, which has a similar sorption potential as Methane, but is generally assumed to be inert, adsorbs onto the surfaces of the mesoporous and microporous regions of the shale samples according to a Langmuir-type behavior. Helium expansion experiments, on the full-diameter core, were used to measure the overall porosity, on the basis that He is a non-sorbing and inert gas as compared to Ar. The He expansion experiments, furthermore, allow us to delineate the mass transfer of gas across the inherent hierarchy of pore sizes, including macropores (macro- and microcracks), mesopores and micropores. Similar expansion experiments were also performed with Ar to study the combined impact of mass transfer and sorption. A triple-porosity model (TPM) was utilized to interpret the He expansion experiments with the shale core and to extract (estimate) relevant transport parameters. We report and compare here the diffusivities and permeabilities of the whole core for both He and Ar, as calculated from the modeling and fitting of the experimental data. On the premise that the shale cube is representative of the matrix region of the core, the Ar sorption kinetics from the cube experiments were subsequently combined with the transport parameters extracted from the He experiments to predict the behavior of the Ar expansion test with the full-diameter core. An excellent agreement is observed between the model predictions and the experimental data. The experimental observations and their interpretation indicate that one must be cautious when using Ar to estimate the true porosity and permeability of shales. In addition, we demonstrate that He and Ar probe gases, when used in tandem, can be employed effectively as a tool to characterize shales in terms of mass transfer and sorption dynamics across scales.

04 OIL SHALES AND TAR SANDS↗

Smashing the Stovepipe: Leveraging the GMSEC Open Architecture and Advanced IT Automation to Rapidly Prototype, Develop and Deploy Next-Generation Multi-Mission Ground Systems

Satellite/Payload Ground Systems - Typically highly-customized to a specific mission's use cases - Utilize hundreds (or thousands!) of specialized point-to-point interfaces for data flows / file transfers Documentation and tracking of these complex interfaces requires extensive time to develop and extremely high staffing costs Implementation and testing of these interfaces are even more cost-prohibitive, and documentation often lags behind implementation resulting in inconsistencies down the road With expanding threat vectors, IT Security, Information Assurance and Operational Security have become key Ground System architecture drivers New Federal security-related directives are generated on a daily basis, imposing new requirements on current / existing ground systems - These mandated activities and data calls typically carry little or no additional funding for implementation As a result, Ground System Sustaining Engineering groups and Information Technology staff continually struggle to keep up with the rolling tide of security Advancing security concerns and shrinking budgets are pushing these large stove-piped ground systems to begin sharing resources - I.e. Operational / SysAdmin staff, IT security baselines, architecture decisions or even networks / hosting infrastructure Refactoring these existing ground systems into multi-mission assets proves extremely challenging due to what is typically very tight coupling between legacy components As a result, many "Multi-Mission" ops. environments end up simply sharing compute resources and networks due to the difficulty of refactoring into true multi-mission systems Utilizing continuous integration / rapid system deployment technologies in conjunction with an open architecture messaging approach allows System Engineers and Architects to worry less about the low-level details of interfaces between components and configuration of systems GMSEC messaging is inherently designed to support multi-mission requirements, and allows components to aggregate data across multiple homogeneous or heterogeneous satellites or payloads - The highly-successful Goddard Science and Planetary Operations Control Center (SPOCC) utilizes GMSEC as the hub for it's automation and situational awareness capability Shifts focus towards getting GS to a final configuration-managed baseline, as well as multi-mission / big-picture capabilities that help increase situational awareness, promote cross-mission sharing and establish enhanced fleet management capabilities across all levels of the enterprise.

GMSEC↗

Optimizing Error-Bounded Lossy Compression for Scientific Data With Diverse Constraints

Vast volumes of data are produced by today's scientific simulations and advanced instruments. These data cannot be stored and transferred efficiently because of limited I/O bandwidth, network speed, and storage capacity. Error-bounded lossy compression can be an effective method for addressing these issues: not only can it significantly reduce data size, but it can also control the data distortion based on user-defined error bounds. In practice, many scientific applications have specific requirements or constraints for lossy compression, in order to guarantee that the reconstructed data are valid for post hoc analysis. For example, some datasets contain irrelevant data that should be isolated in particular and users often have intuition regarding value ranges, geospatial regions, and other data subsets that are crucial for subsequent analysis. Existing state-of-the-art error-bounded lossy compressors, however, do not consider these constraints during compression, resulting in inferior compression ratios with respect to user's post hoc analysis, due to the fact that the data itself provides little or no value for post hoc analysis. In this work we address this issue by proposing an optimized framework that can preserve diverse constraints during the error-bounded lossy compression, e.g., cleaning the irrelevant data, efficiently preserving different precision for multiple value intervals, and allowing users to set diverse precision over both regular and irregular regions. We perform our evaluation on a supercomputer with up to 2,100 cores. Experiments with six real-world applications show that our proposed diverse constraints based error-bounded lossy compressor can obtain a higher visual quality or data fidelity on reconstructed data with the same or even higher compression ratios compared with the traditional state-of-the-art compressor SZ. Furthermore, our experiments also demonstrate very good scalability in compression performance compared with the I/O throughput of the parallel file system.

97 MATHEMATICS AND COMPUTING↗

Challenges in Long-Term Records Preservation and Access - 19549

The US Department of Energy Office of Legacy Management (LM) mission includes preserving records and information about the nation's Cold War-era weapons projects and making that information available for years to come. As LM assumes responsibility for each project, records documenting operations and cleanup activities are absorbed into a central recordkeeping system. Each record category is governed by a federal retention requirement. Routine administrative records are kept just a few years, while the most important records have 'permanent' retention requirements and are transferred to the National Archives typically within 25 years of creation. It is relatively easy for LM to meet retention requirements for these routine and permanent records. LM's big preservation challenge is with records with tremendously long retention requirements. Records with 75-year retention requirements are common, and a small subset of LM records pertaining to waste disposal requires 150-year retention periods. LM must consider the best format for maintaining each of its records collections. Should physical records be boxed up for long-term storage, or should they be digitized? Each option has associated benefits, costs, and risks. Keeping records in paper format avoids digitizing expense and preserves the original media, but the paper must be maintained in a certified storage facility. A vermin infestation or sprinkler malfunction at a certified facility may be rare but could easily destroy records. Keeping the paper and making reference or backup electronic copies reduces the risk of loss but requires diligence and resources to keep the physical records and electronic versions synchronized. This becomes an overwhelming task in a database with millions of records. Entirely replacing paper records with digitized versions as electronic records introduces the risk that records could be lost due to a cybersecurity incident or made inaccessible by evolving technology. LM's early electronic records management efforts were limited to documents saved to PDF format. More recently, LM has embraced the benefits of maintaining records in native formats. (A spreadsheet includes formulas and other details lost when converted to a PDF file.) But whether the file is saved as a Microsoft Excel spreadsheet or PDF document, how do we ensure the file is readable in 75 or 150 years when these records meet their retention requirement? We have no way of knowing for sure from our 2019 vantage point-no more than we would expect to get good technology advice from someone 150 years ago. We cannot predict the technological future, but we can make sure our current electronic records environment is healthy and future ready. This includes making sure the electronic recordkeeping system is built on a modern platform and ensuring that each record in the system is saved in a current format. Quality standards are critical for electronic records to be future ready. Implementing quality involves an organization-wide approach to having consistently formatted file names and complete and descriptive metadata. Quality PDF documents are scanned to settings that ensure characters are machine-readable (with optical character recognition, or OCR). LM is currently replacing its electronic recordkeeping system and simultaneously modernizing its organizational approach to data management; it is replacing a traditional electronic records repository with a system that also manages records stored anywhere across the computer network. This new records management approach will require new thinking in how LM manages a dramatically increased volume of information across the network while not losing focus on its responsibility for long-term records preservation and access. (author)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗