The idea in plain language.
A sequential network evaluation can sometimes be solved through parallel fixed-point updates.
How it works
The paper expresses the intermediate activations of a network as a system of equations. Instead of evaluating every dependency in strict order, iterative solvers update candidate activations, sometimes in parallel. The approach studies Jacobi, Gauss-Seidel, and hybrid updates and evaluates whether their parallel work can reduce elapsed computation time.
What to keep in mind
Benefits depend on convergence, available parallel hardware, and the original computation. Parallel iterations do not guarantee a speedup for every architecture.
Source: Accelerating Feedforward Computation via Parallel Nonlinear Equation Solving. The original manuscript contains the methods, experiments, figures, and references. An arXiv posting date may follow an earlier conference publication. Read the linked record for version history.