Engineering Papers⌕ Search

Engineering topics

Zhang, Weiping

Publications and source records attributed to Zhang, Weiping.

7 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

An Efficient Checkpointing System for Large Machine Learning Model Training

As machine learning models increase in size and complexity rapidly, the cost of checkpointing in ML training became a bottleneck in storage and performance (time). For example, the latest GPT-4 model has massive parameters at the scale of 1.76 trillion. It is highly time and storage consuming to frequently writes the model to checkpoints with more than 1 trillion floating point values to storage. This work aims to understand and attempt to mitigate this problem. First, we characterize the checkpointing interface in a collection of representative large machine learning/language models with respect to storage consumption and performance overhead. Second, we propose the two optimizations: i) A periodic cleaning strategy that periodically cleans up outdated checkpoints to reduce the storage burden; ii) A data staging optimization that coordinates checkpoints between local and shared file systems for performance improvement.

machine learning, artificial intelligence↗
DOE OSTI2024

Distributed Order Recording Techniques for Efficient Record-and-Replay of Multi-threaded Programs

After all these years and all these other shared memory programming frameworks, OpenMP is still the most popular one. However, its greater levels of non-deterministic execution makes debugging and testing more challenging. The ability to record and deterministically replay the program execution is key to address this challenge. However, scalably replaying OpenMP programs is still an unresolved problem. In this paper, we propose two novel techniques that use Distributed Clock (DC) and Distributed Epoch (DE) recording schemes to eliminate excessive thread synchronization for OpenMP record and replay. Our evaluation on representative HPC applications with ReOMP, which we used to realize DC and DE recording, shows that our approach is 2-5x more efficient than traditional approaches that synchronize on every shared-memory access. Furthermore, we demonstrate that our approach can be easily combined with MPI-level replay tools to replay non-trivial MPI+OpenMP applications. We achieve this by integrating ReOMP into ReMPI, an existing scalable MPI record-and-replay tool, with only a small MPI-scale-independent runtime overhead.

Fu, Xiang↗
DOE OSTI2024

Benchmarking Variables for Checkpointing in HPC Applications

Checkpoint/Restart (C/R) is a widely used fault tolerance mechanism in converged systems of cloud, edge, and HPC. However, users often rely on their experience to determine which variables to checkpoint, as there is currently no benchmark that can provide a reference. This can result in checkpointing redundant or even incorrect variables. To address this issue, we propose a benchmark suite that includes critical variables for checkpointing, which have been manually identified, and a method for identifying those critical variables, with 20 representative HPC applications. Our method involves analyzing data dependency between variables to identify critical variables analytically. We verify the identified variables' correctness with a widely used C/R library FTI by an ablation study. With our benchmark suite and data dependency analysis, HPC practitioners now have a reference for identifying checkpointing variables and better knowledge of what kind of variables to checkpoint.

Fu, Xiang↗
DOE OSTI2021

Oleaginous microorganisms and uses of same

Recombinant microorganisms including one or more modifications that enhance lipid production and uses of same for producing lipid. The modifications alter cell surface functions, envelope functions, and other functions. The modifications are capable of converting non-oleaginous microorganisms into oleaginous microorganisms. The lipids produced by some of the recombinant microorganisms are excreted.

Donohue, Timothy James↗
DOE OSTI2018

Methods of processing aromatic compounds

A method of processing a solution comprising aromatic compounds. The method includes culturing a first microorganism in the solution for a time sufficient to reduce an amount of an aromatic compound and thereby generate a processed solution. The culturing may remove an aromatic compound deleterious to growth of a second microorganism without substantially reducing fermentable sugars, thereby permitting enhanced growth of the second microorganism in the processed solution. The culturing may additionally or alternatively convert an aromatic compound into a commodity chemical. The methods of the present invention are advantageous for processing lignocellulosic biomass for upgrading to biofuel or for generating commodity chemicals therefrom.

Noguera, Daniel R,↗
DOE OSTI2017

Mutations That Alter the Bacterial Cell Envelope Increase Lipid Production

ABSTRACT Lipids from microbes offer a promising source of renewable alternatives to petroleum-derived compounds. In particular, oleaginous microbes are of interest because they accumulate a large fraction of their biomass as lipids. In this study, we analyzed genetic changes that alter lipid accumulation in Rhodobacter sphaeroides . By screening an R. sphaeroides Tn 5 mutant library for insertions that increased fatty acid content, we identified 10 high-lipid (HL) mutants for further characterization. These HL mutants exhibited increased sensitivity to drugs that target the bacterial cell envelope and changes in shape, and some had the ability to secrete lipids, with two HL mutants accumulating ~60% of their total lipids extracellularly. When one of the highest-lipid-secreting strains was grown in a fed-batch bioreactor, its lipid content was comparable to that of oleaginous microbes, with the majority of the lipids secreted into the medium. Based on the properties of these HL mutants, we conclude that alterations of the cell envelope are a previously unreported approach to increase microbial lipid production. We also propose that this approach may be combined with knowledge about biosynthetic pathways, in this or other microbes, to increase production of lipids and other chemicals. IMPORTANCE This paper reports on experiments to understand how to increase microbial lipid production. Microbial lipids are often cited as one renewable replacement for petroleum-based fuels and chemicals, but strategies to increase the yield of these compounds are needed to achieve this goal. While lipid biosynthesis is often well understood, increasing yields of these compounds to industrially relevant levels is a challenge, especially since genetic, synthetic biology, or engineering approaches are not feasible in many microbes. We show that altering the bacterial cell envelope can be used to increase microbial lipid production. We also find that the utility of some of these alterations can be enhanced by growing cells in bioreactor configurations that can be used industrially. We propose that our findings can inform current and future efforts to increase production of microbial lipids, other fuels, or chemicals that are currently derived from petroleum.

59 BASIC BIOLOGICAL SCIENCES↗