Binomial Checkpointing for Arbitrary Programs with No User Annotation

Many computing applications require the derivative of a function for a certain process or computation. One of the more efficient methods for evaluating the derivative of a function is automatic differentiation (AD), which has two forms, forward mode and reverse mode. For practical programs that rely on many derivatives, both forms can be unwieldly and complex.

Researchers at Purdue University have developed a modification of the commonly used automatic differentiation form, reverse mode, which does not require user annotation, known as checkpointing reverse mode. Checkpointing reverse mode offers a practical compromise between the existing forward and reverse modes of AD and only increases the runtime and space by a logarithmic factor. This technique can be implemented to any arbitrary program because it is written so the code is completely interchangeable with ordinary reverse mode. In addition, this program, unlike existing checkpointing reverse modes, can automatically and dynamically introduce checkpoints instead of requiring user intervention.

Advantages:

-No user annotation required

-Can apply to nearly any program

-Increased efficiency

Potential Applications

-Computer engineering

-Optimization programs

-Machine learning

TRL: 5

Intellectual Property:

Keywords: Automatic differentiation, AD, checkpointing reverse mode, reverse mode, forward mode, computer engineering, optimization programs, machine learning, derivative evaluation, dynamic checkpointing, Computer Programming, Computer Science, Computer Technology, Machine Learning, Optimization