SEARCH · Engineering Papers

Results for “machine vision”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

304 records · Page 17

All sources NASA NTRS DOE OSTI

NASA NTRS2003

The H-Metaphor as a Guideline for Vehicle Automation and Interaction

Good design is not free of form. It does not necessarily happen through a mere sampling of technologies packaged together, through pure analysis, or just by following procedures. Good design begins with inspiration and a vision, a mental image of the end product, which can sometimes be described with a design metaphor. A successful example from the 20th century is the desktop metaphor, which took a real desktop as an orientation for the manipulation of electronic documents on a computer. Initially defined by Xerox, then refined by Apple and others, it could be found on almost every computer by the turn of the 20th century. This paper sketches a specific metaphor for the emerging field of highly automated vehicles, their interactions with human users and with other vehicles. In the introduction, general questions on vehicle automation are raised and related to the physical control of conventional vehicles and to the automation of some late 20th century vehicles. After some words on design metaphors, the H-Metaphor is introduced. More details of the metaphor's source are described and their application to human-machine interaction, automation and management of intelligent vehicles sketched. Finally, risks and opportunities to apply the metaphor to technical applications are discussed.

Flemisch, Frank O.↗

NASA NTRSDate not supplied

Developing Methods for Exercise System Kinematic Tracking

BACKGROUND How to quantify the load and forces produced by exercise equipment and their Vibration Isolation and Stabilization (VIS) platforms in-flight is an active area of investigation. Kinematic tracking paired with system modeling can provide insights as well as verification and validation of simulations used for system design and development. Traditional motion capture methods can require significant cost in equipment procurement and crew-time, but newer lessons learned can be leveraged [1]. The VIS systems of current and future exercise hardware on the International Space Station (ISS) such as the Cycle Ergometer with Vibration Isolation System (CEVIS) and the European Enhanced Exploration Exercise Device (E4D) are not currently outfitted with IMUs or similar measurement devices. Video-based methods would enable use of multi-purpose, crew-familiar flight equipment. An initial exploration of video-based solutions was performed utilizing 2-camera video from crew cycling on Teal-CEVIS on the ISS. METHODS AND RESULTS Our group has scoped a variety of video-based object tracking methods. To date, we have primarily investigated computer vision toolkits such as open CV. Techniques explored include key-point detection, background subtraction, region-of interest tracking, color-based tracking, tag masking and tracking, and corner detection. Although object-tracking and 6D pose estimation is a rich field, space applications are a unique problem that are challenging for existing software and toolkits. The majority of the existing object-tracking applications involve vehicles/pedestrians and household objects with simple backgrounds. We have identified the following features which pose particular challenges for on-station exercise equipment tracking: 1. Busy and visually cluttered background 2. Low-textured tracking object with relatively small motions 3. Occlusions and motion by human subject and loose, floating objects 4. Limited number of video cameras with no fixed global references 5. Limited ability to add tags, markers, or visual references to the tracking object 6. Lack of training data for Machine Learning (ML) algorithms CONCLUSION We will summarize the efficacy of techniques tested for a ground mock-trial and the on-station exercise trial. It is likely that human-in-loop feedback or a conglomerate of methods is required. ML-based methods, like those implemented for human body tracking [2], may still be a viable option, but more training data and validation is needed.

L Nilsson↗

NASA NTRS1991

The Space Station Freedom Flight Telerobotic Servicer: the design and evolution of a dexterous space robot

The Flight Telerobotic Servicer (FTS) Project at the Goddard Space Flight Center is developing an advanced telerobotic system to assist in and reduce crew extravehicular activity (EVA) for Space Station) Freedom (SSF). The FTS will provide a telerobotic capability to the Freedom Station in the early assembly phases of the program and will be employed for assembly, maintenance, and inspection applications throughout the lifetime of the space station. Appropriately configured elements of the FTS will also be employed for robotic manipulation in remote satellite servicing applications and possibly the Lunar/Mars Program. In mid-1989, the FTS entered the flight system design and implementation phase (Phase C/D) of development with the signing of the FTS prime contract with Martin Marietta Astronautics Group in Denver, Colorado. The basic FTS design is now established and can be reported on in some detail. This paper will describe the FTS flight system design and the rationale for the specific design approaches and component selections. The current state of space technology and the nature of the FTS task dictate that the FTS be designed with sophisticated teleoperation capabilities for its initial primary operating mode. However, there are technologies, such as advanced computer vision and autonomous planning techniques currently in research and advanced development phases which would greatly enhance the FTS capabilities to perform autonomously in less structured work environments. Therefore, a specific requirement on the initial FTS design is that it has the capability to evolve as new technology becomes available. This paper will describe the FTS design approach for evolution to more autonomous capabilities. Some specific task applications of the FTS and partial automation approaches of these tasks will also be discussed in this paper.

Task Performance and Analysis↗

NASA NTRS2008

The Photogrammetry Cube

We can determine distances between objects and points of interest in 3-D space to a useful degree of accuracy from a set of camera images by using multiple camera views and reference targets in the camera s field of view (FOV). The core of the software processing is based on the previously developed foreign-object debris vision trajectory software (see KSC Research and Technology 2004 Annual Report, pp. 2 5). The current version of this photogrammetry software includes the ability to calculate distances between any specified point pairs, the ability to process any number of reference targets and any number of camera images, user-friendly editing features, including zoom in/out, translate, and load/unload, routines to help mark reference points with a Find function, while comparing them with the reference point database file, and a comprehensive output report in HTML format. In this system, scene reference targets are replaced by a photogrammetry cube whose exterior surface contains multiple predetermined precision 2-D targets. Precise measurement of the cube s 2-D targets during the fabrication phase eliminates the need for measuring 3-D coordinates of reference target positions in the camera's FOV, using for example a survey theodolite or a Faroarm. Placing the 2-D targets on the cube s surface required the development of precise machining methods. In response, 2-D targets were embedded into the surface of the cube and then painted black for high contrast. A 12-inch collapsible cube was developed for room-size scenes. A 3-inch, solid, stainless-steel photogrammetry cube was also fabricated for photogrammetry analysis of small objects.

Source record↗

NASA NTRSDate not supplied

DRF: A Software Architecture for a Data Marketplace to Support Advanced Air Mobility

Advanced Air Mobility is a new aviation vision, where unmanned aerial systems will trans- port passengers and cargo across urban and rural areas. Critical to the realization of this vision is the development of a digital marketplace, which allows service providers and consumers operating in the airspace ecosystem to securely exchange data and reasoning insights. In this paper, we present the architecture of a decentralized data marketplace that connects data and reasoning service providers to vehicles and other service consumers along the cloud-to-edge continuum. We also present two example use cases to demonstrate the value of our approach.

Autonomy↗

NASA NTRS1989

Detection of edges using local geometry

Researchers described a new representation, the local geometry, for early visual processing which is motivated by results from biological vision. This representation is richer than is often used in image processing. It extracts more of the local structure available at each pixel in the image by using receptive fields that can be continuously rotated and that go to third order spatial variation. Early visual processing algorithms such as edge detectors and ridge detectors can be written in terms of various local geometries and are computationally tractable. For example, Canny's edge detector has been implemented in terms of a local geometry of order two, and a ridge detector in terms of a local geometry of order three. The edge detector in local geometry was applied to synthetic and real images and it was shown using simple interpolation schemes that sufficient information is available to locate edges with sub-pixel accuracy (to a resolution increase of at least a factor of five). This is reasonable even for noisy images because the local geometry fits a smooth surface - the Taylor series - to the discrete image data. Only local processing was used in the implementation so it can readily be implemented on parallel mesh machines such as the MPP. Researchers expect that other early visual algorithms, such as region growing, inflection point detection, and segmentation can also be implemented in terms of the local geometry and will provide sufficiently rich and robust representations for subsequent visual processing.

Gualtieri, J. A.↗

NASA NTRS2007

Ares Launch Vehicles Overview: Space Access Society

America is returning to the Moon in preparation for the first human footprint on Mars, guided by the U.S. Vision for Space Exploration. This presentation will discuss NASA's mission, the reasons for returning to the Moon and going to Mars, and how NASA will accomplish that mission in ways that promote leadership in space and economic expansion on the new frontier. The primary goals of the Vision for Space Exploration are to finish the International Space Station, retire the Space Shuttle, and build the new spacecraft needed to return people to the Moon and go to Mars. The Vision commits NASA and the nation to an agenda of exploration that also includes robotic exploration and technology development, while building on lessons learned over 50 years of hard-won experience. NASA is building on common hardware, shared knowledge, and unique experience derived from the Apollo Saturn, Space Shuttle, and contemporary commercial launch vehicle programs. The journeys to the Moon and Mars will require a variety of vehicles, including the Ares I Crew Launch Vehicle, which transports the Orion Crew Exploration Vehicle, and the Ares V Cargo Launch Vehicle, which transports the Lunar Surface Access Module. The architecture for the lunar missions will use one launch to ferry the crew into orbit, where it will rendezvous with the Lunar Module in the Earth Departure Stage, which will then propel the combination into lunar orbit. The imperative to explore space with the combination of astronauts and robots will be the impetus for inventions such as solar power and water and waste recycling. This next chapter in NASA's history promises to write the next chapter in American history, as well. It will require this nation to provide the talent to develop tools, machines, materials, processes, technologies, and capabilities that can benefit nearly all aspects of life on Earth. Roles and responsibilities are shared between a nationwide Government and industry team. The Exploration Launch Projects Office at the Marshall Space Flight Center manages the design, development, testing, and evaluation of both vehicles and serves as lead systems integrator. A little over a year after it was chartered, the Exploration Launch Projects team is testing engine components, refining vehicle designs, performing wind tunnel tests, and building hardware for the first flight test of Ares I-X, scheduled for spring 2009. The Exploration Launch Projects team conducted the Ares I System Requirements Review (SRR) at the end of 2006. In Ares' first year, extensive trade studies and evaluations were conducted to refine the design initially recommended by the Exploration Systems Architecture Study, conceptual designs were analyzed for fitness, and the contractual framework was assembled to enable a development effort unparalleled in American space flight since the Space Shuttle. Now, the project turns its focus to the Preliminary Design Review (PDR), scheduled for 2008. Taking into consideration the findings of the SRR, the design of the Ares I is being tightened and refined to meet the safety, operability, reliability, and affordability goals outlined by the Constellation Program. The Ares V is in the early design stage, focusing its activities on requirements validation and ways to develop this heavy-lift system so that synergistic hardware commonality between it and the Ares I can reduce the operational footprint and foster sustained exploration across the decades ahead.

Cook, Steve↗

NASA NTRS1995

Helicopter Human Factors

Even under optimal conditions, helicopter flight is a most demanding form of human-machine interaction, imposing continuous manual, visual, communications, and mental demands on pilots. It is made even more challenging by small margins for error created by the close proximity of terrain in NOE flight and missions flown at night and in low visibility. Although technology advances have satisfied some current and proposed requirements, hardware solutions alone are not sufficient to ensure acceptable system performance and pilot workload. However, human factors data needed to improve the design and use of helicopters lag behind advances in sensor, display, and control technology. Thus, it is difficult for designers to consider human capabilities and limitations when making design decisions. This results in costly accidents, design mistakes, unrealistic mission requirements, excessive training costs, and challenge human adaptability. NASA, in collaboration with DOD, industry, and academia, has initiated a program of research to develop scientific data bases and design principles to improve the pilot/vehicle interface, optimize training time and cost, and maintain pilot workload and system performance at an acceptable level. Work performed at Ames, and by other research laboratories, will be reviewed to summarize the most critical helicopter human factors problems and the results of research that has been performed to: (1) Quantify/model pilots use of visual cues for vehicle control; (2) Improve pilots' performance with helmet displays of thermal imagery and night vision goggles for situation awareness and vehicle control; (3) Model the processes by which pilots encode maps and compare them to the visual scene to develop perceptually and cognitively compatible electronic map formats; (4) Evaluate the use of spatially localized auditory displays for geographical orientation, target localization, radio frequency separation; (5) Develop and flight test control/display concepts; (6) Quantify, model, predict, and improve pilots, workload-management strategies; and (7) Design computer-game trainers to reduce training time and cost.

Hart, Sandra G.↗

NASA NTRS2007

Designing for Our Future in Space

Over the past several years, the disciplines of architecture and human factors have been increasingly recognized as specialties that have focused upon "human-centered design" in the development of spacecraft and surface habitats. These specialties have been instrumental in the conceptual design of overall spacecraft configurations and layouts, as well as habitability outfitting hardware, such as the galley, hygiene facility, sleep quarters, or the layout of displays and controls. From the human-centered perspective, this approach to design assists in the mitigation of risk when designing for an extreme environment such as space. It takes into account the human s physical and cognitive capabilities and limitations, the human s performance in the context of human space flight, the human s interaction with machines that are both physically and cognitively complex, the activities required of the human to accomplish the goals of missions, and the use of design practices that promote products to enable human activity. It is this latter aspect - the use of design practices that promote products to enable human activity - that is the focus of the approach used by the Rhode Island School of Design (RISD) in collaboration with the Habitability and Human Factors Branch (HHFB) at the NASA Johnson Space Center (JSC). During the past few years, there has been a growing recognition of the value added by utilizing industrial designers to further the conceptual development of space hardware, that when used in conjunction with architecture and human factors, provides a robust solution to the design challenge. The "Design for Extreme Environments" Studio at RISD has taken suggested design topics from the NASA JSC HHFB and asked the students to investigate solutions to these challenges. The topics have demanded that the student pay particular attention to a variety of aspects of the space environment and understand how the human responds to each. The student must then adapt the design to these responses. The studio environment has been one way to introduce these challenges, but providing for an "in-residence" opportunity at JSC has given the students a broader vision and set of experiences. The accompanying presentation highlights the studio as well as in-residence work that has been accomplished.

Connolly, Janis↗

NASA NTRS2011

Lunabotics Mining Competition: Inspiration Through Accomplishment

NASA's Lunabotics Mining Competition is designed to promote the development of interest in space activities and STEM (Science, Technology, Engineering, and Mathematics) fields. The competition uses excavation, a necessary first step towards extracting resources from the regolith and building bases on the moon. The unique physical properties of lunar regolith and the reduced 1/6th gravity, vacuum environment make excavation a difficult technical challenge. Advances in lunar regolith mining have the potential to significantly contribute to our nation's space vision and NASA space exploration operations. The competition is conducted annually by NASA at the Kennedy Space Center Visitor Complex. The teams that can use telerobotic or autonomous operation to excavate a lunar regolith geotechnical simulant, herein after referred to as Black Point-1 (or BP-1) and score the most points (calculated as an average of two separate 10-minute timed competition attempts) will eam points towards the Joe Kosmo Award for Excellence and the scores will reflect ranking in the on-site mining category of the competition. The minimum excavation requirement is 10.0 kg during each competition attempt and the robotic excavator, referred to as the "Lunabot", must meet all specifications. This paper will review the achievements of the Lunabotics Mining Competition in 2010 and 2011, and present the new rules for 2012. By providing a framework for robotic design and fabrication, which culminates in a live competition event, university students have been able to produce sophisticated lunabots which are tele-operated. Multi-disciplinary teams are encouraged and the extreme sense of accomplishment provides a unique source of inspiration to the participating students, which has been shown to translate into increased interest in STEM careers. Our industrial sponsors (Caterpillar, Newmont Mining, Harris, Honeybee Robotics) have all stated that there is a strong need for skills in the workforce related to robotics and automated machines. In 2010, 22 United States (US) universities competed, and in May 2011 the competition was opened to international participation, with 46 Universities attending. There were 12 international teams and 34 US teams. This combined total directly inspired an estimated 544 university students. More students and the public were engaged via internet broadcasting and social networking media. This is expected to be of value for actual future space missions, as knowledge is gained from testing many innovative prototypes in simulated lunar regolith. More information is available at www.nasa.gov/lunabotics/.

Mueller, Robert P.↗

NASA NTRS2006

Launching to the Moon, Mars, and Beyond

The U.S. Vision for Space Exploration, announced in 2004, calls on NASA to finish constructing the International Space Station, retire the Space Shuttle, and build the new spacecraft needed to return to the Moon and go on the Mars. By exploring space, America continues the tradition of great nations who mastered the Earth, air, and sea, and who then enjoyed the benefits of increased commerce and technological advances. The progress being made today is part of the next chapter in America's history of leadership in space. In order to reach the Moon and Mars within the planned timeline and also within the allowable budget, NASA is building upon the best of proven space transportation systems. Journeys to the Moon and Mars will require a variety of vehicles, including the Ares I Crew Launch Vehicle, the Ares V Cargo Launch Vehicle, the Orion Crew Exploration Vehicle, and the Lunar Surface Access Module. What America learns in reaching for the Moon will teach astronauts how to prepare for the first human footprints on Mars. While robotic science may reveal information about the nature of hydrogen on the Moon, it will most likely tale a human being with a rock hammer to find the real truth about the presence of water, a precious natural resource that opens many possibilities for explorers. In this way, the combination of astronauts using a variety of tools and machines provides a special synergy that will vastly improve our understanding of Earth's cosmic neighborhood.

Dumbacher, Daniel L.↗

NASA NTRS1991

Lunar crane system

In many lunar construction scenarios, mechanical cranes in some form will be indispensible in moving large masses around with various degrees of fine positioning. While thorough experience exists in the use of terrestrial cranes new thinking is required about the design of cranes to be used in extraterrestrial construction. The primary driving force for this new thinking is the need to automate the crane system so that space cranes can be operated as telerobotic machines with a large number of automatic capabilities. This is true because in extraterrestrial construction human resources will need to be critically rationed. The design problems of mechanisms and control systems for a lunar crane must deal with at least two areas of performance. First, the automated crane must be capable of maneuvering a large mass, so that when the mass arrives at the target position there are only small vibrations. Secondly, any residue vibrations must be automatically damped out and a fine positioning must be achieved. For extraterrestrial use there are additional challenges to a crane design - for example, to design a crane system so that it can be transformed for other construction uses. This initial project in crane design does not address such additional issues, although they may be the subject of future CSC research. To date the Center has designed and analyzed many mechanisms. The fundamental problem of trade-offs between passively stabilizing the load and actively controlling the load by actuators was extensively studied. The capability of 3D dynamics modeling now exists for such studies. A scaled model of a lunar crane was set up and it has been most fruitful in providing basic understanding of lunar cranes. Due to an interesting scaling match-up, this scaled model exhibits the load vibration frequencies one would expect in the real lunar case. Using the analytical results achieved to date, a laboratory crane system is now being developed as a test bed for verifying a wide variety of mechanisms and control designs. Future development will be aimed at making the crane system a telerobotic test bed into which external sensors such as computer vision systems, and other small robotic devices such as CSC lunar rovers, will be integrated.

Mikulas, Martin M., Jr.↗

NASA NTRS1999

Dreams, Hopes, Realities: NASA's Goddard Space Flight Center, the First Forty Years

Throughout history, the great achievements of civilizations and cultures have been recorded in lists of dates and events. But to look only at the machinery, discoveries, or milestones is to miss the value of these achievements. Each goal achieved or discovery or made represents a supreme effort on the part of individual people who came and worked together for a purpose greater than themselves. Driven by an innate curiosity of the spirit, we have built civilizations and discovered new worlds, always reaching out beyond what we knew or thought was possible. These efforts may have used ships or machinery, but the achievement was that of the humans who made those machines possible- remarkable people willing to endure discomfort, frustration, fatigue, and the risk of failure in the hope of finding out something new. This is the case with the history of the Goddard Space Flight Center. This publication traces the legacy of successes, risks, disappointments and internationally recognized triumphs of the Center's first 40 years. It is a story of technological achievement and scientific discovery; of reaching back to the dawn of time and opening up a new set of eyes on our own planet Earth. In the end, it is not a story about machinery or discoveries, but a story about ourselves. If we were able to step off our planet, and if we continue to discover new mysteries and better technology, it is because the people who work at Goddard always had a passion for exploration and the dedication to make it happen. The text that follows is a testimony to the challenges people at the Goddard Space Flight Center have faced and overcome over almost half a century. Today, we stand on the threshold of a new and equally challenging era. It will once again test our ingenuity, skills, and flexibility as we find new ways of working with our colleagues in industry, government, and academia. Doing more with less is every bit as ambitious as designing the first science instrument to study the heavens. But if we are to continue exploring our world and our universe, it is every bit as important. Robert H. Goddard once said, "The dream of yesterday is the hope of today and the reality of tomorrow." This is our heritage. Our challenge is to keep our spirit of dedication, vision, and innovative thinking alive, so we can turn today's dreams into a new century of possibility and progress. Our journeys into space are the greatest ongoing adventure the human race has ever undertaken, and everyone here has played an important role in that endeavor. This book is about everyone who has worked at the Goddard Space Flight Center.

Wallace, Lane E.↗

NASA NTRSDate not supplied

Real World Applications of AI/ML in Optimizing Airspace Operations

As National Airspace System (NAS) is going through the Digital Transformation journey, data science and analytics methods can significantly contribute to improving the traditional physics-based decision-making tools. The adoption of AI/ML methods will not only help accelerate Federal Aviation Administration (FAA)’s vision of Info-centric NAS but also contribute to the overall objective of sustainable aviation. AI/ML can improve the ground and airspace operations by enhancing the accuracy of current decision-making tools used by the airlines and the FAA to manage traffic on the ground and in the air. Huge amount of data that gets collected during a flight. AI/ML methods can extract information from this data and provide valuable insights to make better operational decisions. NASA has partnered with the FAA and commercial airlines such as American and Southwest Airlines on this effort and has successfully demonstrated the benefits of using ML in real world environment by reducing delays and optimizing ground operations at the US airports. In 2022 itself, NASA demonstrated real-world benefits (over 24K lbs. of fuel savings, over 76.6K lbs. CO2 emission savings, and several hours of delay savings) by deploying ML based prediction models to optimize ground operations at Dallas/Fort Worth International and Dallas Love Field Airports in Texas. These tools are being deployed on the cloud for broader deployment, adaptability, and scalability. NASA is developed a reference implementation of the cloud-based platform to significantly lower the bar to development and distribution of these digital services for aviation. In this talk, I will share information about the Digital Information Platform project, the novel AI/ML based approaches used for optimizing ground operations and the opportunities to partner with NASA on these demonstrations.

air traffic management↗

NASA NTRS2018

NASA Pilot-Engaged Expert Response Using IBM Watson Technology: Prototype Evaluation of Knowledge Retrieval System

NASA Langley Research Center and IBM have been investigating the use of IBM Watson technology in aerospace research and development. One application of Watson technology is the Pilot-Engaged Expert Response (PEER) use case. The PEER system is envisioned as an in-cockpit advisor that will act as a source of situationally-relevant information for pilots and other flight crew members to assist in decision making about real-time events and situations that arise in the course of aircraft operations. PEER will make available vast stores of knowledge and information quickly and directly, putting important informational resources where they are needed most. IBM has worked with NASA to develop an architecture and articulate a roadmap for the development of the PEER system. That vision is built around Watson Discovery Advisor (WDA) software solution, derived from IBM's Jeopardy!-winning automatic question answering system. PEER makes use of WDA's sophisticated question-answering capabilities as its core, adding important User Interface components and other customizations for the cockpit environment, including communication with flight systems and other external data sources. The development plan for PEER includes four development stages, with the current project constituting the first phase. In this project, a prototype instance of PEER was successfully adapted to the aviation domain, enabling users to ask questions about aviation topics and receive useful and accurate answers to these questions. Major tasks accomplished include the development of procedures for domain adaptation through automatic lexicon extraction from domain glossaries; generation of question-answer training data which was used to train the system; and assessment of the effectiveness of domain adaptation, which showed a dramatic improvement in the ability of the PEER system to answer domain-relevant questions. In addition, the vision for the PEER system was pushed forward by the articulation of a plan for the automatic enhancement of question-answering with contextual information. This initial phase focused on two main goals: 1) the targeted domain adaptation of the underlying WDA system to the aviation domain; and, 2) the design of the software systems needed to leverage flight-contextual data. Domain adaptation of the WDA system proceeds via three main activities: Domain data ingestion, lexical customization and model training. A textual corpus consisting of 1,147 individual documents with more than 7.5 million words of text was ingested into the system and this served as the basis of all further development. A domain lexicon of over 3,500 aviation-domain terms was semi-automatically generated from domain documents and used to train the system. In addition, a set of over 500 question-answer (QA) pairs relevant to the PEER use case was developed; these were used to train and assess the system. These important first steps established the basis for the PEER system. In addition, steps were taken towards the integration of the PEER system into the cockpit environment with the development of a functional design for the Contextual Data Augmentation (CDA) subsystem. This subsystem brings to bear contextual data to improve system responses. It has three main submodules: the Contextual Data Collection module, the Contextual Data Selection module, and the Contextual QA Augmentation module. These modules form a processing pipeline that addresses the problems associated with automatically integrating information from external resources into the knowledge-retrieval mechanism.

Machine learning↗

NASA NTRS2024

Artificial Intelligence Medical Support for Long-Duration Space Missions

We envision an artificial intelligence (AI) based system that will provide support and recommendations to the crew medical officer (CMO) and ground flight surgeon during long-duration space missions. Such a system would be pretrained on the knowledgebase of clinical knowledge on Earth, minimizing the amount of Earth data that needs to be transferred into space. Then during deployment, the system would be constantly refined through active learning from diverse streams of data from sensors in the spacecraft, data collected daily from individual astronauts, and human-in-the-loop feedback from the crew. The model could be interrogated for predictions and recommendations on personalized crew health based on the overall status of the spacecraft, medicinal stores, and status of other crew members. Adaptation techniques would be used to incorporate spaceflight data that have very different distributions from the training data due to the extreme environment. Edge computing and the most advanced neuromorphic processing would enable computation in scenarios with low power and bandwidth, while dimensionality reduction would be employed to ensure that the input data streams from spaceflight are as small as possible. In order to realize this long-term vision, several hardware and software aspects need to be developed and assembled. First, models pretrained on Earth biomedical data would need to be evaluated for predictive accuracy, and the best one selected. That model would need to be adapted to learn from diverse, sparse, and inconsistently measured data streams, as well as human-in-the-loop feedback. A data integration, standardization, and dimensionality reduction methodology would need to be developed to handle all data types and feed them into the model. Once the software and data infrastructure is developed, it would need to be integrated with small footprint compute processors and tested in high-radiation, high-vibration, unregulated temperature situations. As a short-term goal, we recommend to focus on the development of the data and model software structure. Several large language models (LLM) already exist that have been trained on Earth biomedical and clinical knowledgebases, including BioMedLLM, Med-PaLM, SPOKE LLM, and Foresight. These models need to be evaluated for accuracy and the best one chosen for a proof-of-concept structure, while maintaining awareness of the accelerating AI field and incorporating any newly improved model architectures as needed. Then, we recommend to develop a database of synthetic data types to mimic the diverse data streams that are expected in a long-duration space mission. This should include environmental and microbial data from the spacecraft, non-invasive data from wearables and point-of-care devices employed by astronauts, and more invasive molecular and physiological monitoring of clinical and biomarker data from astronauts. The data standardization methodology should be developed, and these data streams used to refine the clinical LLM. Several scenarios should be developed that could plausibly come up in a long-duration space mission, and changes or aberrations introduced to the data at specific times to mimic these scenarios. Then, question and answer tasks should be designed to interrogate the model for predictions and recommendations, with acceptable answers already identified.

Artificial Intelligence↗