Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Federated Transfer Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Potential nanotechnology applications for reducing freshwater consumption at coal fired power plants : an early view.

This report was funded by the U.S. Department of Energy's (DOE's) National Energy Technology Laboratory (NETL) Existing Plants Research Program, which has an energy-water research effort that focuses on water use at power plants. This study complements the overall research effort of the Existing Plants Research Program by evaluating water issues that could impact power plants. A growing challenge to the economic production of electricity from coal-fired power plants is the demand for freshwater, particularly in light of the projected trends for increasing demands and decreasing supplies of freshwater. Nanotechnology uses the unique chemical, physical, and biological properties that are associated with materials at the nanoscale to create and use materials, devices, and systems with new functions and properties. It is possible that nanotechnology may open the door to a variety of potentially interesting ways to reduce freshwater consumption at power plants. This report provides an overview of how applications of nanotechnology could potentially help reduce freshwater use at coal-fired power plants. It was developed by (1) identifying areas within a coal-fired power plant's operations where freshwater use occurs and could possibly be reduced, (2) conducting a literature review to identify potential applications of nanotechnology for facilitating such reductions, and (3) collecting additional information on potential applications from researchers and companies to clarify or expand on information obtained from the literature. Opportunities, areas, and processes for reducing freshwater use in coal-fired power plants considered in this report include the use of nontraditional waters in process and cooling water systems, carbon capture alternatives, more efficient processes for removing sulfur dioxide and nitrogen oxides, coolants that have higher thermal conductivities than water alone, energy storage options, and a variety of plant inefficiencies, which, if improved, would reduce energy use and concomitant water consumption. These inefficiencies include air heater inefficiencies, boiler corrosion, low operating temperatures, fuel inefficiencies, and older components that are subject to strain and failure. A variety of nanotechnology applications that could potentially be used to reduce the amount of freshwater consumed - either directly or indirectly - by these areas and activities was identified. These applications include membranes that use nanotechnology or contain nanomaterials for improved water purification and carbon capture; nano-based coatings and lubricants to insulate and reduce heat loss, inhibit corrosion, and improve fuel efficiency; nano-based catalysts and enzymes that improve fuel efficiency and improve sulfur removal efficiency; nanomaterials that can withstand high temperatures; nanofluids that have better heat transfer characteristics than water; nanosensors that can help identify strain and impact damage, detect and monitor water quality parameters, and measure mercury in flue gas; and batteries and capacitors that use nanotechnology to enable utility-scale storage. Most of these potential applications are in the research stage, and few have been deployed at coal-fired power plants. Moving from research to deployment in today's economic environment will be facilitated with federal support. Additional support for research development and deployment (RD&D) for some subset of these applications could lead to reductions in water consumption and could provide lessons learned that could be applied to future efforts. To take advantage of this situation, it is recommended that NETL pursue funding for further research, development, or deployment for one or more of the potential applications identified in this report.

01 COAL, LIGNITE, AND PEAT↗

Lessons Learned from FUSRAP

The US DOE Office of Legacy Management (LM) is the long-term steward for 90 sites remediated under numerous regulatory regimes including the Formerly Utilized Sites Remedial Action Program (FUSRAP) sites. In addition, LM holds considerable historical information, gathered in the 1970s, to determine site eligibility for remediation under FUSRAP. To date, 29 FUSRAP sites are in LM’s inventory of sites for long-term surveillance and maintenance (LTS&M), and 25 are with the US Army Corps of Engineers (USACE) for remediation or in the process of being transitioned to LM. It is forecasted that 13 FUSRAP sites will transfer from the USACE to LM over the next 10 years; however, the timing of the transfers is strongly dependent upon federal funding of the ongoing remedial actions. Historically, FUSRAP sites were generally cleaned up for “unrestricted” industrial use or remediated to the “cleanup standards” at that time, and their use remained unchanged. Today, these sites as well as the adjacent properties are now changing or envisioned to have changes in land use, typically from industrial to commercial or residential uses. The implication of land-use change affects DOE’s LTS&M responsibility for the sites under LM stewardship as well as the planning for the additional sites scheduled to transition in time. Coinciding with land-use changes at or near FUSRAP sites is an increased community awareness of these sites. As property development increases near FUSRAP sites, the general public and interested stakeholders regularly inquire about the sufficiency of cleanups that impact their neighborhoods and communities. LM has used this experience to address a series of lessons learned to improve our program management in light of the changing conditions of our sites. We describe these lessons learned as (1) improved stakeholder relations, (2) enhanced LTS&M requirements for the sites, and (3) greater involvement in the transition process.

54 ENVIRONMENTAL SCIENCES↗

Lessons Learned from FUSRAP - 16200

The US DOE Office of Legacy Management (LM) is the long-term steward for 90 sites remediated under numerous regulatory regimes including the Formerly Utilized Sites Remedial Action Program (FUSRAP) sites. In addition, LM holds considerable historical information, gathered in the 1970's, to determine site eligibility for remediation under FUSRAP. To date, 29 FUSRAP sites are in LM's inventory of sites for long-term surveillance and maintenance (LTS and M), and 25 are with the US Army Corps of Engineers (USACE) for remediation or in the process of being transitioned to LM. It is forecasted that 13 FUSRAP sites will transfer from the USACE to LM over the next 10 years; however, the timing of the transfers is strongly dependent upon federal funding of the ongoing remedial actions. Historically, FUSRAP sites were generally cleaned up for unrestricted industrial use or remediated to the cleanup standards at that time, and their use remained unchanged. Today, these sites as well as the adjacent properties are now changing or envisioned to have changes in land use, typically from industrial to commercial or residential uses. The implication of land-use change affects DOE's LTS and M responsibility for the sites under LM stewardship as well as the planning for the additional sites scheduled to transition in time. Coinciding with land-use changes at or near FUSRAP sites is an increased community awareness of these sites. As property development increases near FUSRAP sites, the general public and interested stakeholders regularly inquire about the sufficiency of cleanups that impact their neighborhoods and communities. LM has used this experience to address a series of lessons learned to improve our program management in light of the changing conditions of our sites. We describe these lessons learned as (1) improved stakeholder relations, (2) enhanced LTS and M requirements for the sites, and (3) greater involvement in the transition process. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Electronic Handbooks Simplify Process Management

Getting a multitude of people to work together to manage processes across many organizations for example, flight projects, research, technologies, or data centers and others is not an easy task. Just ask Dr. Barry E. Jacobs, a research computer scientist at Goddard Space Flight Center. He helped NASA develop a process management solution that provided documenting tools for process developers and participants to help them quickly learn, adapt, test, and teach their views. Some of these tools included editable files for subprocess descriptions, document descriptions, role guidelines, manager worksheets, and references. First utilized for NASA's Headquarters Directives Management process, the approach led to the invention of a concept called the Electronic Handbook (EHB). This EHB concept was successfully applied to NASA's Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) programs, among other NASA programs. Several Federal agencies showed interest in the concept, so Jacobs and his team visited these agencies to show them how their specific processes could be managed by the methodology, as well as to create mockup versions of the EHBs.

Source record↗

Predicting U.S. federal fleet electric vehicle charging patterns using internal combustion engine vehicle fueling transaction statistics

Utilizing fueling transactions from internal combustion engine vehicles (ICEVs), the authors estimated how frequently midday public charging would be required for U.S. federal fleet battery electric vehicles (BEVs). Fueling transaction summary statistics are more widely available than trip-level telematics data, making this methodology more accessible and transferable to other researchers and fleet managers considering BEV replacements. For example, readers can easily apply a linear model using only the count of back-to-back fueling events at gas stations over 57 straight-line miles apart to predict days exceeding range. This linear regression predicted binned days exceeding 250 miles at 80% accuracy on a hold-out test set from the same fleet as the training data and 66 % accuracy on a new fleet displaying different driving behaviors. The authors additionally provide linear equations for days exceeding 200 and 300 miles as alternative range estimates to account for differences in BEV range and temperature impacts. Beyond the single-feature linear models which readers can apply, the authors tuned and trained other machine learning models on a variety of fueling transaction statistics including consecutive transaction distances, transaction distance from garage, estimated miles traveled from fuel economy and fuel quantity, and transaction periodicity. Utilizing a subset of 1678 light-duty federal fleet vehicles which contained daily vehicle miles traveled (VMT) in addition to fueling statistics, the authors determined which fueling transaction statistics were most relevant in predicting driving days exceeding 250 miles (an approximation of BEV rated driving range). In support of the U.S. federal fleet transition to zero-emission vehicles (ZEVs), the authors used these statistics and machine learning models to predict the frequency of BEV midday charging. After training models on the subset with VMT, the authors predicted days exceeding rated range for 112,902 light-duty vehicles operating in similar circumstances in the federal fleet using a Support Vector Regressor (SVR). In conclusion, they then used the projections as part of the ZEV Planning and Charging (ZPAC) tool to identify optimal candidates for BEVs for the federal fleet. An anonymized version of ZPAC is included in the supplementary materials.

25 ENERGY STORAGE↗

Summary of Observations from the 2017 Federal Radiological Monitoring and Assessment Center (FRMAC) Laboratory Analysis Training and Capstone Event

On August 15th thru 17th, 2017 the Federal Radiological Monitoring and Assessment Center (FRMAC) Laboratory Analysis division, the FRMAC Fly Away Laboratory (FAL), the FRMAC Assessment division, and the Mobile Environmental Response Laboratory (MERL) held a training and capstone event for staff from the Environmental Protection Agency (EPA), Remote Sensing Laboratory (RSL), Lawrence Livermore National Laboratory (LLNL), and Sandia National Laboratories (SNL). LAB-100, "Sample Control Training", LAB-200 "QA Specialist Training", and LAB-300 "Laboratory Analysis Manager Training" was given the first two days of the event. The purpose of the training and capstone event was to meet training requirements for billeted FRMAC Lab Analysis staff from RSL, LLNL, and SNL as well as raise awareness of the FRMAC Lab Analysis process with EPA staff as part of a Federal Emergency Management Agency — Nuclear Incident Response Team (FEMA-NIRT) laboratory standardization project. An objective of the standardization project was to help improve the transition of operations from DOE to EPA during a response. To do this effectively, detailed knowledge of the FRMAC Lab Analysis process by the EPA is needed. This training provided a good opportunity for this knowledge transfer. A capstone was held after the two-day training event to allow participants to practice the skills they learned in a realistic scenario. A scenario that was previously developed for a quarterly Consequence Management drill (i.e. Dark Phoenix) was used as the basis for the capstone, with laboratory analysis focused injects used to drive the exercise play. Each position within the FRMAC Lab Analysis Division exercised to specific objectives and helped to uncover gaps in the established processes. The lessons learned during this capstone are broken out in the following categories: Sample Control, In-Situ Gamma Spectroscopy, Analysis Request Forms (ARF), Shipping, QA/QC, Fly Away Laboratory (FAL), and Management.

54 ENVIRONMENTAL SCIENCES↗

NASA/DOD Aerospace Knowledge Diffusion Research Project: Survey of Reader Preferences Concerning the Format of NASA Langley-Authored Technical Reports - Paper 58

The U.S. government technical report is a primary means by which the results of federally funded research and development (R&D) are transferred to the U.S. aerospace industry. However, little is known about this information product in terms of its actual use, importance, and value in the transfer of federally funded R&D. Little is also known about the intermediary-based system that is used to transfer the results of federally funded R&D to the U.S. aerospace industry. To help establish a body of knowledge, the U.S. government technical report is being investigated as part of the NASA/DOD Aerospace Knowledge Diffusion Research Project. In this paper, we summarize the literature on the U.S. government technical report and present the results of a survey of U.S. aerospace engineers and scientists that solicited their opinions concerning the format of NASA Langley Research Center (LaRC)-authored technical reports. To learn more about the preferences of U.S. aerospace engineers and scientists concerning the format of NASA LaRC-authored technical reports, we surveyed 133 report producers (i.e., authors) and 137 report users in March-April 1996. Questions covered such topics as (1) the order in which report components are read, (2) components used to determine if a report would be read, (3) those components that could be deleted, (4) the placement of such components as the symbols list, (e) the de-sirability of a table of contents, (5) the format of reference citations, (6) column layout and right margin treatment, and (7) and person and voice. Mail (self-reported) surveys were used to collect the data. The response rates for report producers (i.e., authors) was 68% and for users was 62%.

Pinelli, Thomas E.↗

NASA/DOD Aerospace Knowledge Diffusion Research Project. Paper 65: Survey of Reader Preferences Concerning the Format of NASA Langley-Authored Technical Reports

The U.S. government technical report is a primary means by which the results of federally funded research and development (R&D) are transferred to the U.S. aerospace industry. However, little is known about this information product in terms of its actual use, importance, and value in the transfer of federally funded R&D. Little is also known about the intermediary-based system that is used to transfer the results of federally funded R&D to the U.S. aerospace industry. To help establish a body of knowledge, the U.S. government technical report is being investigated as part of the NASA/DoD Aerospace Knowledge Diffusion Research Project. In this article, we summarize the literature on the U.S. government technical report and present the results of a survey of U.S. aerospace engineers and scientists that solicited their opinions concerning the format of NASA Langley Research Center (LaRC)-authored technical reports. To learn more about the preferences of U.S. aerospace engineers and scientists concerning the format of NASA LaRC-authored technical reports, we surveyed 133 report producers (i.e., authors) and 137 report users in March-April 1996. Questions covered such topics as: (a) the order in which report components are read; (b) components used to determine if a report would be read; (c) those components that could be deleted; (d) the placement of such components as the symbols list; (e) the desirability of a table of contents; (f) the format of reference citations; (g) column layout and right margin treatment; and (h) writing style in terms of person and voice. Mail (self-reported) surveys were used to collect the data. The response rates for report producers (i.e., authors) was 68% and for users was 62%.

Pinelli, Thomas E.↗

DEVELOP’s Approach to Experiential Learning

The NASA DEVELOP Program addresses environmental decision making needs and geoscience workforce development through 10-week feasibility studies that apply Earth observations to environmental issues at hand. The program builds capacity to use geospatial information in both its participants (students, recent graduates, early career professionals, and transitioning career professionals) and partner organizations (federal agencies, state & local governments, non-profits, and private industry). This is accomplished through a structured project execution model that provides opportunities for participants to have autonomy, learn “on the job,” and gain new skillsets for working with remote sensing data. A pipeline of leadership positions enhances opportunities for individuals engaged in the program to get hands-on experience conducting data analyses, communicating their work, leading technical projects, and building their knowledge bank of Earth-observing satellite capabilities. These skillsets and knowledge are then transferred to partners through the projects. This panel contribution will introduce the DEVELOP model, highlight experiences of participants, and share the program’s insights into good practices for effective experiential learning.

Capacity Building↗

State Technologies Advancement Collaborative

The U. S. Department of Energy (DOE), National Association of State Energy Officials (NASEO), and Association of State Energy Research and Technology Transfer Institutions (ASERTTI) signed an intergovernmental agreement on November 14, 2002, that allowed states and territories and the Federal Government to better collaborate on energy research, development, demonstration and deployment (RDD&D) projects. The agreement established the State Technologies Advancement Collaborative (STAC) which allowed the states and DOE to move RDD&D forward using an innovative competitive project selection and funding process. A cooperative agreement between DOE and NASEO served as the contracting instrument for this innovative federal-state partnership obligating funds from DOE's Office of Energy Efficiency and Renewable Energy and Office of Fossil Energy to plan, fund, and implement RDD&D projects that were consistent with the common priorities of the states and DOE. DOE's Golden Field Office provided Federal oversight and guidance for the STAC cooperative agreement. The STAC program was built on the foundation of prior Federal-State efforts to collaborate on and engage in joint planning for RDD&D. Although STAC builds on existing, successful programs, it is important to note that it was not intended to replace other successful joint DOE/State initiatives such as the State Energy Program or EERE Special Projects. Overall the STAC process was used to fund, through three competitive solicitations, 35 successful multi-state research, development, deployment, and demonstration projects with an overall average non-federal cost share of 43%. Twenty-two states were awarded at least one prime contract, and organizations in all 50 states and some territories were involved as subcontractors in at least one STAC project. Projects were funded in seven program areas: (1) Building Technologies, (2) Industrial Technologies, (3) Transportation Technologies, (4) Distributed Energy Resources, (5) Hydrogen Technology Learning Centers, (6) Fossil Energy, and (7) Rebuild America.

01 COAL, LIGNITE, AND PEAT↗

Catalyzing deep decarbonization with federated battery diagnosis and prognosis for better data management in energy storage systems

Industrial data analytics methods play a central role in improving energy storage performance and efficiency, impacting the future of electrified transportation and renewable electricity generation. However, significant challenges hinder the large-scale deployment of batteries. Conventional methods rely on centralized collection and processing of fleet-level data, leading to database size issues and privacy concerns due to potential data breaches. To enable scalable deployment of battery management systems, this article proposes a federated battery diagnosis and prognosis model, which distributes the processing of battery standard current-voltage-time-usage data in a privacy-preserving manner. Instead of transferring the raw data, this approach communicates only the locally processed parameters, thus reducing communication load and preserving data confidentiality. The federated model offers a paradigm shift in battery health management through privacy-preserving distributed methods for battery data processing and lifetime prediction, ensuring the reliable and sustainable deployment of lithium-ion batteries in a rapidly evolving world.

asset health management↗

University of Utah, Energy Commercialization Center

During the Energy Commercialization Center’s (ECC) three years in operation, the only thing constant was change. The world of commercialization and cleantech evolved significantly during the time the ECC was formed and operating, including: the availability of cleantech funding lessoned, the growth of incubators and accelerators skyrocketed, the State of Utah created an office dedicated to energy development, the University of Utah was both praised and criticized for its success in commercialization, and the Federal government temporarily shut down. During the three-year grant there were three principle investigators on the grant, as well as three directors for the University’s Commercialization Office. Change can be hard for an organization,but as we instruct the companies we support, “Fail fast and fail often, because it is the fastest path to success.” Although there were some unanticipated challenges along the way, the local ecosystem is stronger because of the ECC’s efforts. Perhaps the greatest lesson learned was the importance of aligned incentives between key stakeholders in the commercialization process and the need for resources at the company and individual entrepreneur levels. The universities have systems and incentives to commercialize technologies, but creating value and companies generally rest with the individuals and entrepreneurs. Unfortunately the ECC was unable to create a viable mechanism to transfer the commercialization process that successfully aligned incentives and achieve a more effective ecosystem within the Rocky Mountain West. However, the ECC was successful in adding value to the individual ecosystems, and connecting national resources to regional and local needs. Regarding the ECC’s effectiveness in developing a cleantech commercialization ecosystem, initial inroads and relationships were established with key stakeholders. However, incentives, perceived or real competition, differences in commercialization processes, and culture all played a role in inhibiting the development and distribution of a regional ecosystem and commercialization process. Had the University and the ECC been able to develop a software platform, some of these challenges may have been overcome, but without the final development and release of the Western Innovation Network, the ECC realistically could not scale and distribute a commercialization platform. Further, cleantech startups need to engage in a more intensive customer validation process, and establish strong community connections if they are to succeed in commercializing their products. The university system incentivizes research and access to research funding and risk capital is competitive, so by nature collaboration on commercialization was difficult. Each of the local ecosystems within the Rocky Mountain West was unique. Utah did not, and does not, have a system outside of the universities to support entrepreneurs and cleantech commercialization. Through the ECC’s efforts developing a regional ecosystem, it became clear that successful ecosystems had a community and associated mechanisms that supported local entrepreneurs and startups. Most importantly the ECC aided in the creation of Utah’s cleantech ecosystem, one that supports entrepreneurs and startup companies that need help and support in their efforts to commercialize clean technologies. The absence of support for clean tech from state government and local organizations was a significant impediment to cleantech commercialization. To overcome this challenge, the ECC has formed Sustainable Startups. Sustainable Startups is a new non-profit organization designed to build a culture and community in Utah that supports and understands the importance of cleantech and sustainable development. While the ECC generated mixed success in building a regional commercialization ecosystem for cleantech, the organization did provide tremendous benefit to startups and the broader public. Over 60 companies were given direct business development support by the ECC, many of whom then generated direct economic development impacts. In addition, the ECC served an important role as community convener, educator and relationship builder through hosting numerous public and private events including: Energize 2013; Millennial Train whistle stop; business plan competition supporter; Clean Tech Open Accelerator organizer; Sustainable Startups Series developer, and much more. While the ECC did not fully apply, develop, and transmit the University of Utah’s TCO commercialization model to cleantech, it nevertheless assisted numerous inventors, entrepreneurs and institutions in furthering the growth of clean energy and energy efficiency technologies. The TCO’s commercialization model was not applied to regional clean tech initiatives for several main reasons. First, flaws with the commercialization model were realized after the ECC’s formation. Second, leadership changes within the TCO and ECC hampered early organizational development and implementation initiatives. Third, misaligned incentives between the ECC, regional universities, institutions, and the State of Utah resulted in a lack of collaboration and knowledge transfer regarding commercialization. In principle, everyone was aligned and willing to collaborate, but reality was much different and challenging.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Spinoff 2003: 100 Years of Powered Flight

Today, NASA continues to reach milestones in space exploration with the Hubble Telescope, Earth-observing systems, the Space Shuttle, the Stardust spacecraft, the Chandra X-Ray Observatory, the International Space Station, the Mars rovers, and experimental research aircraft these are only a few of the many initiatives that have grown out of NASA engineering know-how to drive the Agency s missions. The technical expertise gained from these programs has transferred into partnerships with academia, industry, and other Federal agencies, ensuring America stays capable and competitive. With Spinoff 2003, we once again highlight the many partnerships with U.S. companies that are fulfilling the 1958 Space Act stipulation that NASA s vast body of scientific and technical knowledge also benefit mankind. This year's issue showcases innovations such as the cochlear implant in health and medicine, a cockpit weather system in transportation, and a smoke mask benefiting public safety; many other products are featured in these disciplines, as well as in the additional fields of consumer/home/recreation, environment and resources management, computer technology, and industrial productivity/ manufactacturing technology. Also in this issue, we devote an entire section to NASA s history in the field of flight and showcase NASA s newest enterprise dedicated to education. The Education Enterprise will provide unique teaching and learning experiences for students and teachers at all levels in science, technology, engineering, and mathematics. The Agency also is committed, as never before, to engaging parents and families through NASA s educational resources, content, and opportunities. NASA s catalyst to intensify its focus on teaching and learning springs from our mission statement: to inspire the next generation of explorers as only NASA can.

Source record↗

Technology Transfer: Marketing Tomorrow's Technology

The globalization of the economy and the end of the Cold War have triggered many changes in the traditional practices of U.S. industry. To effectively apply the resources available to the United States, the federal government has firmly advocated a policy of technology transfer between private industry and government labs, in this case the National Aeronautics and Space Administration (NASA). NASA Administrator Daniel Goldin is a strong proponent of this policy and has organized technology transfer or commercialization programs at each of the NASA field centers. Here at Langley Research Center, the Technology Applications Group (TAG) is responsible for facilitating the transfer of Langley developed research and technology to U.S. industry. Entering the program, I had many objectives for my summer research with TAG. Certainly, I wanted to gain a more thorough understanding of the concept of technology transfer and Langley's implementation of a system to promote it to both the Langley community and the community at large. Also, I hoped to become more familiar with Langley's research capabilities and technology inventory available to the public. More specifically, I wanted to learn about the technology transfer process at Langley. Because my mentor is a member of Materials and Manufacturing marketing sector of the Technology Transfer Team, another overriding objective for my research was to take advantage of his work and experience in materials research to learn about the Advanced Materials Research agency wide and help market these developments to private industry. Through the various projects I have been assigned to work on in TAG, I have successfully satisfied the majority of these objectives. Work on the Problem Statement Process for TAG as well as the development of the Advanced Materials Research Brochure have provided me with the opportunity to learn about the technology transfer process from the outside looking in and the inside looking out. Because TAG covers all of the research efforts conducted at Langley, my studies with TAG were ab!e to provide me an excellent overview of Langley's contribution to the aeronautics industry.

Tcheng, Erene↗

CAFE AU LAIT: Compute-Aware Federated Augmented Low-Rank AI Training

Federated finetuning is crucial for unlocking the knowledge embedded in pretrained Large Language Models (LLMs) when data are geographically distributed across clients. Unlike finetuning with data from a single institution, federated finetuning allows collaboration across multiple institutions, enabling the utilization of diverse and decentralized datasets while preserving data privacy. Given the high computing costs of LLM training and the emphasis on energy efficiency in Federated Learning (FL), Low-Rank Adaptation (LoRA) has emerged as a widely adopted algorithm due to its significantly reduced number of trainable parameters. However, this assumes that all data silos have the necessary computing resources to compute local updates of LLMs. Nevertheless, in practice, the computing resources across clients are highly heterogeneous: while some may have access to hundreds of GPUs, others might have limited or no GPU access. Recently, federated finetuning using synthetic data has been proposed, allowing clients to participate in a collaborative training run without training LLMs locally. However, our experimental results reveal a performance gap between models trained using synthetic data and those trained using local updates. Motivated by the observed heterogeneity in computing resources and the performance gap, we propose a novel two-stage algorithm that leverages the storage and computing capabilities of a strong server. In the first stage, under the coordination of the strong server, clients with limited computing resources collaborate to generate synthetic data, which is transferred to and stored on the strong server. In the second stage, the strong server uses this synthetic data on behalf of the resource-constrained clients to perform federated LoRA finetuning alongside clients with sufficient computing resources. This approach ensures that all clients can participate in the finetuning process. Experimental results demonstrate that incorporating local updates from even a small fraction of clients improves performance compared to using synthetic data for all clients. Furthermore, we incorporate the Gaussian mechanism in both stages to guarantee client-level differential privacy.

Wang, Jiayi [ORNL]↗

Towards an Aviation Large Language Model by Fine-tuning and Evaluating Transformers

In the aviation domain, there are many applications for machine learning and artificial intelligence tools that utilize natural language. For example, there is a desire to know the commonalities in written safety reports such as voluntary post incidents reports or create more accurate transcripts of air traffic management conversations. Another use-case is the possibility of extracting airspace procedures and constraints currently written in documents such as Letters of Agreement (LOA) which is used as the evaluation case in this paper. These applications can benefit from the use of state-of-the-art Natural Language Processing (NLP) techniques when adapted to the language/phraseology specific to the aviation domain. This paper evaluates the viability of transferring pre-trained large language models to the aviation domain by adapting transformer based models using aviation datasets. This paper utilized two datasets to adapt a ‘Robustly Optimized Bidirectional Encoder Representations from Transformers Approach’ (RoBERTa) model and two down-stream classification tasks to assess its performance. These datasets are all built upon Letters of Agreement which are Federal Aviation Administration (FAA) documents that formalize airspace operations across the national airspace system. The first two datasets are used for the adaptation of RoBERTa to the aviation domain and were of different sizes to assess the number of documents needed to adapt to the aviation domain. They contain many examples of ‘aviation English’ using domain specific terminology and phrasing which serves as a representative basis to perform the unsupervised adaptation. The second dataset is a separate set of LOA documents with two sets of classification labels to be used for evaluation; one at the document level and one at the line level. These down-stream evaluations allowed the measurement of improvement by adapting RoBERTa. The accuracy increased by 4-6% on both tasks and the F1 score on the class of interest increased by 4-8% from the adaptation.

Air Traffic Management↗

Towards an Aviation Large Language Model by Fine-tuning and Evaluating Transformers

In the aviation domain, there are many applications for machine learning and artificial intelligence tools that utilize natural language. For example, there is a desire to know the commonalities in written safety reports such as voluntary post incidents reports or create more accurate transcripts of air traffic management conversations. Another use-case is the possibility of extracting airspace procedures and constraints currently written in documents such as Letters of Agreement (LOA) which is used as the evaluation case in this paper. These applications can benefit from the use of state-of-the-art Natural Language Processing (NLP) techniques when adapted to the language/phraseology specific to the aviation domain. This paper evaluates the viability of transferring pre-trained large language models to the aviation domain by adapting transformer based models using aviation datasets. This paper utilized two datasets to adapt a ‘Robustly Optimized Bidirectional Encoder Representations from Transformers Approach’ (RoBERTa) model and two down-stream classification tasks to assess its performance. These datasets are all built upon Letters of Agreement which are Federal Aviation Administration (FAA) documents that formalize airspace operations across the national airspace system. The first two datasets are used for the adaptation of RoBERTa to the aviation domain and were of different sizes to assess the number of documents needed to adapt to the aviation domain. They contain many examples of ‘aviation English’ using domain specific terminology and phrasing which serves as a representative basis to perform the unsupervised adaptation. The second dataset is a separate set of LOA documents with two sets of classification labels to be used for evaluation; one at the document level and one at the line level. These down-stream evaluations allowed the measurement of improvement by adapting RoBERTa. The accuracy increased by 4-6% on both tasks and the F1 score on the class of interest increased by 4-8% from the adaptation.

Air Traffic Management↗

Development of a Preliminary Performance Assessment to Support Single-Shell Tank Waste Retrieval - 19446

A preliminary Performance Assessment (PA) of Single-Shell Tank (SST) Waste Management Area A-AX (WMA A-AX) located at DoE's Hanford Site in southeastern Washington is being conducted to satisfy the requirements of the Hanford Federal Facility Agreement and Consent Order (HFFACO), as well as other Federal and State requirements of closure plans and permits. The WMA A-AX PA effort is assessing the fate, transport, and impacts of radionuclides and hazardous chemicals within residual wastes left in tanks, ancillary equipment, and facilities in their assumed closed configuration, and the subsequent risks to humans into the far future. WMA A-AX includes the 241-A Tank Farm (A Farm), the 241-AX Tank Farm (AX Farm), and the ancillary equipment associated with each Farm. There are six underground SSTs in A Farm, and four underground SSTs in AX Farm. Each of these SSTs have a storage capacity of 1,000,000 gallons. A complex waste transfer system of ancillary equipment supported the transfer and storage of waste within WMA A-AX SSTs. This ancillary equipment includes transfer lines, catch tanks, diversion boxes, vaults, valve pits, and other miscellaneous structures. This paper emphasizes the development of the PA effort focused on radiological impacts to meet the requirements for a closure authorization under DOE Order 435.1. Lessons learned from retrieval and development of a recently completed PA of the nearby Waste Management Area C (WMA C) led to the decision to develop a preliminary PA for WMA A-AX prior to initiation of retrieval efforts so that a site-specific risk management capability could be available to support anticipated retrieval decisions. The technical approach for the PA analysis includes developing both a process-level model using site-specific hydrogeological data, as well as a system-level model. The system model is being developed to facilitate uncertainty and sensitivity analyses being conducted in the PA effort. Preliminary results of the DOE Order 435.1 PA are evaluating potential impacts to the groundwater and air pathways to determine radiation exposure once radionuclides are released to the environment from residual waste after closure of the tank farm. To meet other analysis requirements in DOE Order 435.1, the PA effort must also evaluate potential radiological impacts from inadvertent intruder scenarios into the closed WMA A-AX facility. The results of the PA evaluation will be compared to performance objectives and measures in the DOE Order 435.1 to support retrieval and eventual closure decisions at WMA A-AX. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗