Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “network interfaces”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,801 records · Page 100
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Resilience Design Patterns: A Structured Approach to Resilience at Extreme Scale (V.2.0)

Reliability is a serious concern for future extreme-scale high-performance computing (HPC) systems. Projections based on the current generation of HPC systems and technology roadmaps suggest the prevalence of very high fault rates in future systems. The errors resulting from these faults will propagate and generate various kinds of failures, which may result in outcomes ranging from result corruptions to catastrophic application crashes. Therefore, the resilience challenge for extreme-scale HPC systems requires coordination between various hardware and software technologies that are capable of handling a broad set of fault models at accelerated fault rates. Also, due to practical limits on power consumption in future HPC systems, they are likely to embrace innovative architectures, increasing the levels of hardware and software complexities. Therefore, the techniques that seek to improve resilience must navigate the complex trade-off space between resilience and the overheads to power consumption and performance. While the HPC community has developed various resilience solutions, application-level techniques as well as system-based solutions, the solution space of HPC resilience techniques remains fragmented. There are no formal methods to integrate the various HPC resilience techniques into composite solutions, nor are there methods to holistically evaluate the adequacy and efficacy of such solutions in terms of their protection coverage, and their performance & power efficiency characteristics. Additionally, few implementations of current resilience solutions are portable to newer architectures and software environments that will be deployed on future systems. We developed a new structured approach to the management of HPC resilience using the concept of resilience-based design patterns. In general, a design pattern is a repeatable solution to a commonly occurring problem. We identified the well-known solutions that are commonly used to deal with faults, errors and failures in HPC systems. In the initial design patterns specification (version 1.0), we described the various solutions, which address specific problems in the design of resilient HPC environments, in the form of patterns. Each pattern describes a problem caused by a fault, error or failure event in an HPC environment, and then describes the core of the solution of the problem in such a way that this solution may be adapted to different systems and implemented at different layers of the system stack. The catalog of these resilience design patterns provides designers with a collection of design elements. To construct complete resilience solutions using combinations of various patterns, we defined a framework that enhances HPC designers' understanding of the important constraints and the opportunities for the design patterns to be implemented and deployed at various layers of the system stack. The design framework is also useful for establishing interfaces and mechanisms to coordinate flexible fault management across hardware and software components, as well as to consider the trade-off between performance, resilience, and power consumption when constructing a solution. The resilience design patterns specification version 1.1 included more detailed explanations of the pattern solutions, the context in which the patterns are applicable, and the implications for hardware or software design. It also provided several additional examples and detailed case studies to demonstrate the use of patterns to build realistic solutions. In version 1.2 of the specification document, we have improved the pattern descriptions, including graphical representations of the pattern components. These improvements are largely based on critical comments, feedback and suggestions received from pattern experts and readers of the previous versions of the specification. The pattern classification has been modified to further clarify the relationships between pattern categories. This version of the specification also introduces a pattern language for resilience design patterns. The pattern language presents the patterns in the catalog as a network, revealing the relations among the resilience patterns. The language provides designers with the means to explore alternative techniques for handling a specific fault model that may have different efficiency and complexity characteristics. Using the pattern language also enables the design and implementation of comprehensive resilience solutions as a set of interconnected resilience patterns that can be instantiated across layers of the system stack. The overall goal of this work is to provide hardware and software designers, as well as the users and operators of HPC systems, a systematic methodology for the design and evaluation of resilience technologies in HPC systems that keep scientific applications running to a correct solution in a timely and cost-efficient manner despite frequent faults, errors, and failures of various types. Version 2.0 expands the resilience design pattern classification and catalog to include self-stabilization patterns and reliability, availability and performance models for each structural pattern.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2018

Using the cFS Command and Data Dictionary (CCDD) to Automate Software Development on Habulous

Final paper is attached. The NASA developed Core Flight System (cFS) is a reusable software architecture that has been used on multiple spaceflight missions. By using this framework, missions are able to reuse code from other missions, as well as leverage deployment onto similar computer architectures (i.e. not "reinvent the wheel" on each new mission). The success in the cFS concept can be seen in the large number of projects using cFS at FSW-2018. The Habulous project is an Earth-based testbed, used for hardware and software that may one day be used on a future space habitat unit, with many participating groups from various NASA centers and aerospace organizations around the country. The distributed nature of the various teams mean that defining (and following) an interface definition is critical on the project. Additionally, since various groups use various types of computer hardware (32/64-bit, big/little endian, Linux/VxWorks/Windows) many additional complications exist in interfacing all the various components into a final integrated system. cFS is used on the majority the flight software (FSW) in running in Habulous. But some subsystems have elected to not use cFS, and use a software bridge (called SBN_lib) to interact with the other cFS nodes in Habulous. In order to most efficiently develop the FSW, a central database is used to define and store each message sent by cFS. A Command and Data Dictionary (CDD) is something nearly universal on spacecraft, but as a team we worked to develop the CDD before the SW development was complete, and not treat it like "as built" documentation. To manage the CDD, the cFS Command and Data Dictionary (CCDD) tool was chosen (available from NASA as open source software). The CCDD tool has successfully been used to automate/autocode a large amount of software used on Habulous, as we are hoping to use it to define even more items in the future (time-triggered Ethernet (TTE) network maps, CPU scheduling). Additionally, Habulous has been exploring the use of cFS on wildly heterogeneous CPUs, and how to coordinate all those various machines using/extending the software bus – network (SBN) application in cFS, as well as TTE to coordinate message passing between various synchronized machines. The major topics to be covered in the presentation are: (1) Updating to the CCSDS_v2 extended headers (and using CPU# as subsystem ID). (2) Managing all the message identification numbers for each cFS message sent/received on any of the various CPUs. (3) Using the CCDD information to automatically generate the C-header files that define the structure for all software bus (SB) commands/telemetry messages. (4) Using the CCDD to automatically generate XML Telemetry and Command Exchange (XTCE) files, which streams display production/integration/testing in a web based display architecture (5) Extending/customizing SBN to pass messages among computers on multiple networks. (6) Using "Protobetter" inside SBN to manage different endian-ness/architectures. (7) Using SBN_lib to allow non-cFS node to communicate with cFS nodes. (8) Developing TTE network and schedule tables for all the various CPUs to use.

Hirsh, Robert L.↗
DOE OSTI2023

Blockchain Empowered Provenance Framework for Sensor Identity Management and Data Flow Security in Fossil-based Power Plants

The overall goals of this project are divided into 3 important phases. First, a peer-to-peer SCADA network is to be established where identity profile of sensors from fossil-power plants will be stored in an immutable manner in order to verify data flow integrity in real time. Second, it is aimed to develop a Blockchain-based provenance platform to audit equipment operations along with data and process integrity violations. In this phase, various operational rules will be incorporated in the ledger on behalf of the operating sensor so its abnormal use can be easily detected. In addition, it is aimed to achieve provenance when the rate of transactions is exceedingly high, thus enabling to design a scalable provenance scheme that can work in real time when number of data collecting points or sensor units increase in fossil-based power plants. Finally, a working testbed will be developed to integrate the Blockchain platform with sensor-based fossil power plant comprising of Remote Terminal Units (RTU) and SCADA devices. Using multiple Raspberry-PIs mimicking the role of RTUs that are interfaced with multiple sensors devices will be deployed to serve as a critical component of fossil power plant and the performance metrics such as throughput, latency, bootstrapping time, etc., will be evaluated on the testbed.

20 FOSSIL-FUELED POWER PLANTS↗
DOE OSTI2023

Trust Model System for the Energy Grid of Things Network Communications

Network communication is crucial in the Energy Grid of Things (EGoT). Without a network connection, the energy grid becomes just a power grid where the energy resources are available to the customer uni-directionally. A mechanism to analyze and optimize the energy usage of the grid can only happen through a medium, a communications network, that enables information exchange between the grid participants and the service provider. Security implementers of EGoT network communication take extraordinary measures to ensure the safety of the energy grid, a critical infrastructure, as well as the safety and privacy of the grid participants. With the dynamic nature of network communication of the EGoT, the information provided by the customer or the service provider can be falsified by a malicious attacker. Therefore, a trust model is necessary to monitor any abnormal activities. This paper describes a distributed trust model system that meets the need of the EGoT. This paper describes methods for evaluating and improving the distributed trust model using standard hypothesis testing metrics such as true positive, false positive, true negative, false negative, equal error rate, and F1 score. Example calculations are shown based on generated sample data.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS2016

The IOAG Recommendations on Spacecraft Emergency Cross Support

In 2014, the Inter-agency Operations Advisory Group (IOAG) chartered a multi-agency team effort to study how to best handle spacecraft emergency as part of cross support among network assets. The original intent is to put in place for the first time a process and guidelines to make emergency support as part of a permanent cross-support capability between space agencies. From the perspective of space communications service providers, a few key issues, common to all agencies, concerning the Spacecraft Emergency Cross Support (SECS) have been explored. They are: context of emergency support, provision of emergency support under a cross-support agreement, provision of emergency support with no cross-support agreement, legal and liability issues, response time, support priority, services provided, sustaining emergency support capabilities, and charge for the support. The positions of the various participating agencies with respect to these issues have been collected, analyzed, and finally harmonized to form the recommended IOAG positions. A central challenge most communications service providers are facing is that since the spacecraft emergency is an unplanned critical event, it typically requires fast response to the emergency call, hence lacking an international standard process for the operational interfaces seems to exacerbate the difficulty in providing SECS. For reducing the response time, i.e. from the time of accepting a request for SECS to the readiness for support, it is recommended that a “cross-support emergency system” be established by IOAG member agencies. Along with it, the IOAG core services, just-in-time ground communications line, cross support service management (CSSM), and standard operations procedures (SOP) for operational interfaces form the basic foundation of the “cross support emergency system”. Of the above, new to the cross support conducted thus far in the IOAG community is the concept of the SOP specifically for the interfaces between the service provider and service user during the SECS. Use cases, salient features, and definition of the key operational activities/tasks relevant to the interfaces are addressed by the effort. Underpinning such a system is the availability of the RF license granted by the local authority, at national and/or regional level, for a given ground station to communicate with and track the spacecraft in emergency mode at the uplink and downlink frequencies assigned to that spacecraft. That means it is critical for the IOAG member agencies to obtain a priori all-band licenses (for the entire X-band or S-band) for some, if not all, of their ground stations that are most capable of or likely to provide SECS. It is also recommended that certain prior arrangements be made with the relevant local licensing authorities for a process that will allow expedited authorization to transmit/receive signals to/from the declared spacecraft over the declared ground stations specifically and solely for the emergency case. Our analysis of the SECS has also uncovered a few fundamental programmatic issues. Recognizing any IOAG positions reached on these issues do not necessarily lead to any binding authority, it is recommended that they, along with those key attributes of the cross support emergency system, be explicitly stated as multi-agency guidelines to guide the implementation and provision of the SECS. The paper will present the results of the working group on Spacecraft Emergency Cross Support and, in particular, its findings, products and recommendations. It will also identify the next steps of this work that may include the production of some international standards as an extension of this work, or the process to make this emergency system usable by other spacecraft operators beyond the space agencies.

Tai, Wallace S.↗
NASA NTRS2016

The IOAG Recommendations on Spacecraft Emergency Cross Support

In 2014, the Inter-agency Operations Advisory Group (IOAG) chartered a multi-agency team effort to study how to best handle spacecraft emergency as part of cross support among network assets. The original intent is to put in place for the first time a process and guidelines to make emergency support as part of a permanent cross-support capability between space agencies. From the perspective of space communications service providers, a few key issues, common to all agencies, concerning the Spacecraft Emergency Cross Support (SECS) have been explored. They are: context of emergency support, provision of emergency support under a cross-support agreement, provision of emergency support with no cross-support agreement, legal and liability issues, response time, support priority, services provided, sustaining emergency support capabilities, and charge for the support. The positions of the various participating agencies with respect to these issues have been collected, analyzed, and finally harmonized to form the recommended IOAG positions. A central challenge most communications service providers are facing is that since the spacecraft emergency is an unplanned critical event, it typically requires fast response to the emergency call, hence lacking an international standard process for the operational interfaces seems to exacerbate the difficulty in providing SECS. For reducing the response time, i.e. from the time of accepting a request for SECS to the readiness for support, it is recommended that a “cross-support emergency system” be established by IOAG member agencies. Along with it, the IOAG core services, just-in-time ground communications line, cross support service management (CSSM), and standard operations procedures (SOP) for operational interfaces form the basic foundation of the “cross support emergency system”. Of the above, new to the cross support conducted thus far in the IOAG community is the concept of the SOP specifically for the interfaces between the service provider and service user during the SECS. Use cases, salient features, and definition of the key operational activities/tasks relevant to the interfaces are addressed by the effort. Underpinning such a system is the availability of the RF license granted by the local authority, at national and/or regional level, for a given ground station to communicate with and track the spacecraft in emergency mode at the uplink and downlink frequencies assigned to that spacecraft. That means it is critical for the IOAG member agencies to obtain a priori all-band licenses (for the entire X-band or S-band) for some, if not all, of their ground stations that are most capable of or likely to provide SECS. It is also recommended that certain prior arrangements be made with the relevant local licensing authorities for a process that will allow expedited authorization to transmit/receive signals to/from the declared spacecraft over the declared ground stations specifically and solely for the emergency case. Our analysis of the SECS has also uncovered a few fundamental programmatic issues. Recognizing any IOAG positions reached on these issues do not necessarily lead to any binding authority, it is recommended that they, along with those key attributes of the cross support emergency system, be explicitly stated as multi-agency guidelines to guide the implementation and provision of the SECS. The paper will present the results of the working group on Spacecraft Emergency Cross Support and, in particular, its findings, products and recommendations. It will also identify the next steps of this work that may include the production of some international standards as an extension of this work, or the process to make this emergency system usable by other spacecraft operators beyond the space agencies.

Tai, Wallace S.↗
DOE OSTI2020

Recommendations for Data-in-Transit Requirements for Securing DER Communications

With the adoption of Distributed Energy Resource (DER) interoperability standards, common communication protocols are now being deployed between power system operators and DER devices. In 2018, a revision to the US interconnection and interoperability standard, Institute of Electrical and Electronics Engineers (IEEE) Std. 1547, required DER equipment to have an IEEE 2030.5, IEEE 1815, or SunSpec Modbus communication exchange interface. This change supports the future transition to secure connection and exchange of information between the DER equipment and implementing parties, such as grid operators. Adoption of standardized communication protocols and associated information models is a critical step toward interoperability between power system operators and DER, such as photovoltaic (PV) and energy storage systems. However, security requirements for these standardized communication protocols are not comprehensive, resulting in non-standard and vendor-specific implementation that may leave DER equipment susceptible to cyberattacks. This paper examines the data-in-flight security requirements for standardized DER communication protocols, per IEEE 1547-2018 revision, as it relates to device authentication, key management, and encryption. The state of the art for these security features is also explored, addressing their impact on communication and performance of low-cost single board computers, which are typical of DER devices. In conclusion, a recommendation is provided to adopt a common set of communication requirements, which are intended to achieve interoperability and implement data security over DER network pathways, while ensuring reliable, secure, and real-time information delivery.

42 ENGINEERING↗
DOE OSTI2025

From tides to seasons: How cyclic tidal drivers and plant physiology interact to affect carbon cycling at the terrestrial-estuarine boundary (Final technical report)

Coastal ecosystems are among the most biologically and biogeochemically active and diverse systems on Earth. Because they act as important linkages between terrestrial ecosystems and the open ocean, their incorporation in Earth system models (ESMs) is critical to predict coastal and global responses to environmental changes. However, they vary greatly in the magnitude of tides and the volume and timing of freshwater input from land, making it challenging to model the major biogeochemical reactions that control productivity and greenhouse gas emissions across coastal terrestrial aquatic interfaces (TAIs). Our overall objective was to improve mechanistic process understanding and modeling of tidal wetland hydro-biogeochemistry in coastal TAIs. We established a new flux tower site (Ameriflux US-PLo) in the oligohaline part of the Parker River to continuously monitor ecosystem-scale carbon fluxes under temporally varying salinity conditions. The site is co-located with long-term monitoring plots of the Plum Island Ecosystems LTER project. We installed wells and redox sensors in the marsh interior and creek bank, established biomass monitoring plots and deployed novel optode sensors in both locations. We used this data to parameterize plant-mediated transport in PFLOTRAN and tested the impact of soil heterogeneity on porewater constituents and gas fluxes. We collected observations of root oxygen release with a novel planar optode system in the field. Flux data collected during the measurement period encompasses a large variation in salinity ranging from drought to record precipitation years. We developed a method to extract functional relationships from the flux data using artificial neural networks, identifying salinity thresholds for CH 4 fluxes. Finally, we are using the coupled ELM-PFLOTRAN model to test the impact of antecedent hydrological conditions on the salinity-CH 4 flux relationship. This grant contributed to the professional development of one postdoc, three research assistants and one graduate student. The sensor data has been shared with external collaborators.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

1.2.4.404 - Data-Driven Approach for Hydropower Plant Controller Prototyping Using Remote Hardware in the Loop (DR-HIL)

Real-time prototyping of hydropower plant controls is important for reducing the cost and the risk of field deployment. This project will 1) collect design and operational data from actual hydro plants and 2) use a physics-informed machine learning approach for real-time emulation of hydropower plants, including hydro turbine and hydrodynamics. The data-driven models will be interfaced with digital real-time simulation at NREL’s Flatirons campus for hardware-in-the-loop (HIL) testing of the governor hardware device or controller-HIL (CHIL). The proposed approach will also establish the connectivity based remote CHIL testing capability using real-time data streams from an actual hydro plant. This integrated hydro-plant emulation with CHIL will be used to prototype hydro-governor controls and eventually provide an opportunity to test hydropower integrated with various technologies (e.g. conventional and renewable generation, energy conversion, etc.) as HIL.

controls prototyping↗
NASA NTRS1999

Modeling and Simulation Reliable Spacecraft On-Board Computing

The proposed project will investigate modeling and simulation-driven testing and fault tolerance schemes for Spacecraft On-Board Computing, thereby achieving reliable spacecraft telecommunication. A spacecraft communication system has inherent capabilities of providing multipoint and broadcast transmission, connectivity between any two distant nodes within a wide-area coverage, quick network configuration /reconfiguration, rapid allocation of space segment capacity, and distance-insensitive cost. To realize the capabilities above mentioned, both the size and cost of the ground-station terminals have to be reduced by using reliable, high-throughput, fast and cost-effective on-board computing system which has been known to be a critical contributor to the overall performance of space mission deployment. Controlled vulnerability of mission data (measured in sensitivity), improved performance (measured in throughput and delay) and fault tolerance (measured in reliability) are some of the most important features of these systems. The system should be thoroughly tested and diagnosed before employing a fault tolerance into the system. Testing and fault tolerance strategies should be driven by accurate performance models (i.e. throughput, delay, reliability and sensitivity) to find an optimal solution in terms of reliability and cost. The modeling and simulation tools will be integrated with a system architecture module, a testing module and a module for fault tolerance all of which interacting through a centered graphical user interface.

Park, Nohpill↗
NASA NTRS2003

Web Program for Development of GUIs for Cluster Computers

WIGLAF (a Web Interface Generator and Legacy Application Facade) is a computer program that provides a Web-based, distributed, graphical-user-interface (GUI) framework that can be adapted to any of a broad range of application programs, written in any programming language, that are executed remotely on any cluster computer system. WIGLAF enables the rapid development of a GUI for controlling and monitoring a specific application program running on the cluster and for transferring data to and from the application program. The only prerequisite for the execution of WIGLAF is a Web-browser program on a user's personal computer connected with the cluster via the Internet. WIGLAF has a client/server architecture: The server component is executed on the cluster system, where it controls the application program and serves data to the client component. The client component is an applet that runs in the Web browser. WIGLAF utilizes the Extensible Markup Language to hold all data associated with the application software, Java to enable platform-independent execution on the cluster system and the display of a GUI generator through the browser, and the Java Remote Method Invocation software package to provide simple, effective client/server networking.

Czikmantory, Akos↗
NASA NTRS2011

An Analysis of Near Fields of 34m Antennas of JPL/NASA Deep Space Network

This paper addresses the issue of calculating near fields of the 34m Beam Waveguide (BWG) antennas of the NASA/JPL Deep Space Network (DSN). Calculating the near fields of DSN antennas are of interest in receive mode where the transmitting signals from nearby flying objects such as helicopters and airplanes could interfere with the operation of sensitive RF receiving system of DSN antennas, and in the transmit mode where fields from high-powered DSN antennas interfere with receivers on nearby flying objects, as well as safety considerations for the operators and visitors to the grounds surrounding the antenna sites. A complete and detailed analysis has been performed using PO/PTD techniques, including surface errors and support struts effects. Some results are presented, including comparisons with preliminary field tests.

near fields↗
DOE OSTI2021

Deep Learning for Fish Identification from Sonar Data: CRADA 481 [Abstract only]

To help solve the challenges of hydropower energy production related to the potential for eel injury and mortality from passage through hydropower turbines, we will develop a deep learning method for identifying migrating eels from imaging sonar. This project continues with a prior project conducted by the Pacific Northwest National Laboratory (PNNL) and the Electric Power Research Institute (EPRI) in FY2018-2019. The proposed method employs Convolution Neural Network (CNN), a powerful deep learning method for image classification, to distinguish between images of eels and non-eel moving objects. We propose to collect more laboratory data and add more existing field data to train a powerful deep learning model. In addition to eels and sticks as classified in previous studies, we will add images containing several non-eel fish species and macrophyte mats to the training data. A multi-class classification model will be developed to distinguish these objects. Object detection algorithm will be explored and developed to locate and identify multiple objects in each sonar frame. Motion analysis will be performed to track the movement of objects in sonar video clips. We will also improve the data conversion algorithm so that it can read in both DIDSON and ARIS (both are imaging sonars developed by Sound Metrics Corp) data files and convert them to images with comparably high resolution, regardless of the varying detection ranges in different environments. The developed algorithms will be packaged as a software with a graphic user interface. The software will be evaluated by external collaborators in the field. The developed framework can be generalized for automatic monitoring of fish passage and migration using other imaging sonars like ARIS and will benefit the design and operation of ecologically friendly hydroelectric projects. The developed wavelet and CNN model configuration parameters can potentially be transferred to lamprey detection in similar riverine environments.

13 HYDRO ENERGY↗
DOE OSTI2024

Artificial Intelligence/Machine Learning Technology in Power System Applications

The primary purpose of this report is to provide an overview of the advancement in artificial intelligence and machine learning (AI/ML) technologies and their applications in power systems. It offers a foundation for understanding the transformative role of AI/ML in power systems and aims to stimulate further research and development in this area. This report begins with a historical perspective of AI/ML technologies, then explores their advancement to today’s prominence. The document highlights key contributors to the success of AI/ML technologies, including increased computational power, greater data availability, innovative algorithms, and advanced tools. It further introduces various AI/ML techniques, including supervised, unsupervised and reinforcement learning, graph neural networks, and generative AI. It also emphasizes the critical importance of ensuring the safety, security, and trustworthiness of these AI/ML techniques within this sector. The report reviews the recent representative advancements in various power system applications enhanced by AI/ML techniques, underscoring key developments and their transformative impact as evidenced by numerous studies. It also explores both the opportunities and challenges associated with the application of AI/ML technologies to improve power system applications. While the report extensively covers AI/ML applications in power systems, focusing primarily on the technical and operational aspects, it may not thoroughly explore the sociopolitical, economic, and broader regulatory implications of AI/ML integration in power systems. AI/ML techniques hold significant potential for enhancing power system applications; however, they are not omnipotent. It is crucial to acknowledge their limitations and understand that they may not be able to address all challenges in the power system domain. Various factors must be considered that influence the implementation, adoption, and effectiveness of AI/ML solutions, including but not limited to safety, security, transparency, and trustworthiness. Additionally, the incorporation of advanced human–machine interfaces is essential, as it enables humans to validate the effectiveness of AI/ML solutions while remaining actively engaged, fostering trust in AI/ML deployment. Finally, the report summarizes AI/ML research activities supported by the Department of Energy (DOE) Office of Electricity (OE) through the Advanced Grid Modeling (AGM) program. The work aligns with the interests and mission of DOE-OE AGM, with the report serving as a resource for identifying existing progress and for pinpointing future applications within AI/ML that need further exploration and support.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2004

Intelligent Systems Technologies and Utilization of Earth Observation Data

The addition of raw data and derived geophysical parameters from several Earth observing satellites over the last decade to the data held by NASA data centers has created a data rich environment for the Earth science research and applications communities. The data products are being distributed to a large and diverse community of users. Due to advances in computational hardware, networks and communications, information management and software technologies, significant progress has been made in the last decade in archiving and providing data to users. However, to realize the full potential of the growing data archives, further progress is necessary in the transformation of data into information, and information into knowledge that can be used in particular applications. Sponsored by NASA s Intelligent Systems Project within the Computing, Information and Communication Technology (CICT) Program, a conceptual architecture study has been conducted to examine ideas to improve data utilization through the addition of intelligence into the archives in the context of an overall knowledge building system (KBS). Potential Intelligent Archive concepts include: 1) Mining archived data holdings to improve metadata to facilitate data access and usability; 2) Building intelligence about transformations on data, information, knowledge, and accompanying services; 3) Recognizing the value of results, indexing and formatting them for easy access; 4) Interacting as a cooperative node in a web of distributed systems to perform knowledge building; and 5) Being aware of other nodes in the KBS, participating in open systems interfaces and protocols for virtualization, and achieving collaborative interoperability.

Ramapriyan, H. K.↗
DOE OSTI2025

Datum: A Scientific Metadata Catalog

The data catalog market is currently flooded with a myriad of different products, but none serve the scientific community well. There are cloud-native tools like Databricks, Snowflake,to on-premise solutions like Collibra and Datahub. The common failing of all these tools however, is their inability to serve the scientific data community directly. Most catalogs are targeted towards financial, health, or user data - not sensor or scientific domain data. They also prioritize integrations that often don’t exist or are just starting to be used in the scientific realm - all while ignoring common scientific tools and file types. Datum is a catalog which targets the scientific data directly, including the tools and networks in which those tools are used. We work with the producers and consumers of the data where they are, targeting cloud and on-premise with a focus on classified networks. Datum is an Erlang/Elixir application. Technical Features Note: The features listed below are still under development and may change, slightly, upon final delivery of the product. File Formats - Datum has the ability to read additional metadata and provides processing pipelines for the following file formats: Plain Text, PDF, LaTeX, HTML, Open Document Format (.odt), XML, CSV/TSV (and other standard delimiters), OpenDocument Database and Spreadsheets, Geo-Referenced TIFF, Common Data Format, HDF/HDF5, LabView TDMS, Excel, DeltaTables, Parquet, Apache Iceberg, Apache Hudi and many others. Metadata Collection - Scanners for the local and networked file systems and cloud storage providers. Network integration with common databases such as MSSQL and MySQL. User Plugin System - Users are able to provide either file processing, metadata extraction, or sampling plugins in the programming language of their choice. Authentication/Authorization -: OIDC integration, SCIM provisioning and EntraID integration out of the box. Full user and group management system with a “least privilege” operating mode. Governance - Customizable data governance platform; dictate and enforce required metadata, enforce data embargos, and enforce user agreements and NDAs before data access. Ability to create health checks on data, rejecting abandoned or poorly curated data and automatically removing it from the search index. Ability for users to submit corrections. Search - Semantic search is a first class citizen. No licenses to expensive, external software required. Integrated use of vectors and vector-based search allows for AI agent integration at all levels of operation. Metadata Model - Display and control data’s lineage and connections to other data and data directories. Data is modeled after a filesystem - an organization instantly recognizable and navigable by most any user. CLI and SDK - Ships with a Command Line Interface (CLI) tool and with a fully-featured Python SDK. This allows for rapid and programmatic use of Datum by every level of user. Minimal Infrastructure - Datum ships as a single executable file and can be run on any operating system and most CPU architectures. Datum has no reliance on external databases, search indexing tools, or other outside services - and it runs equally well on edge computing devices, cloud services, or in a clustered HPC environment.

darrington, john↗
NASA NTRS2020

Global earth mineral inventory: A data legacy

Minerals contain important clues to understanding the complex geologic history of Earth and other planetary bodies. Therefore, geologists have been collecting mineral samples and compiling data about these samples for centuries. These data have been used to better understand the movement of continental plates, the oxidation of Earth's atmosphere and the water regime of ancient martian landscapes. Datasets found at ‘RRUFF.info/Evolution’ and ‘mindat.org’ have documented a wealth of mineral occurrences around the world. One of the main goals in geoinformatics has been to facilitate discovery by creating and merging datasets from various scientific fields and using statistical methods and visualization tools to inspire and test hypotheses applicable to modelling Earth's past environments. To help achieve this goal, we have compiled physical, chemical and geological properties of minerals and linked them to the above-mentioned mineral occurrence datasets. As a part of the Deep Time Data Infrastructure, funded by the W.M. Keck Foundation, with significant support from the Deep Carbon Observatory (DCO) and the A.P. Sloan Foundation, GEMI (‘Global Earth Mineral Inventory’) was developed from the need of researchers to have all of the required mineral data visible in a single portal, connected by a robust, yet easy to understand schema. Our data legacy integrates these resources into a digestible format for exploration and analysis and has allowed researchers to gain valuable insights from mineralogical data. GEMI can be considered a network, with every node representing some feature of the datasets, for example, a node can represent geological parameters like colour, hardness or lustre. Exploring subnetworks gives the researcher a specific view of the data required for the task at hand. GEMI is accessible through the DCO Data Portal (https://dx.deepcarbon.net/11121/6200-6954-6634-8243-CC). We describe our efforts in compiling GEMI, the Data Policies for usage and sharing, and the evaluation metrics for this data legacy.

data legacy↗
NASA NTRS2012

Optical Communications Channel Combiner

NASA has identified deep-space optical communications links as an integral part of a unified space communication network in order to provide data rates in excess of 100 Mb/s. The distances and limited power inherent in a deep-space optical downlink necessitate the use of photon-counting detectors and a power-efficient modulation such as pulse position modulation (PPM). For the output of each photodetector, whether from a separate telescope or a portion of the detection area, a communication receiver estimates a log-likelihood ratio for each PPM slot. To realize the full effective aperture of these receivers, their outputs must be combined prior to information decoding. A channel combiner was developed to synchronize the log-likelihood ratio (LLR) sequences of multiple receivers, and then combines these into a single LLR sequence for information decoding. The channel combiner synchronizes the LLR sequences of up to three receivers and then combines these into a single LLR sequence for output. The channel combiner has three channel inputs, each of which takes as input a sequence of four-bit LLRs for each PPM slot in a codeword via a XAUI 10 Gb/s quad optical fiber interface. The cross-correlation between the channels LLR time series are calculated and used to synchronize the sequences prior to combining. The output of the channel combiner is a sequence of four-bit LLRs for each PPM slot in a codeword via a XAUI 10 Gb/s quad optical fiber interface. The unit is controlled through a 1 Gb/s Ethernet UDP/IP interface. A deep-space optical communication link has not yet been demonstrated. This ground-station channel combiner was developed to demonstrate this capability and is unique in its ability to process such a signal.

Quirk, Kevin J.↗

Refine your search to explore more results.