Algorithm now available to exactly compute information rate

This year it is 75 years ago that Claude Shannon, the ’father of information theory’, showed how information transmission can be quantified mathematically, namely via the so-called information transmission rate. Yet, until now this quantity could only be computed approximately. AMOLF researchers Manuel Reinhardt and Pieter Rein ten Wolde, together with a collaborator from Vienna, have now developed a simulation technique that—for the first time—makes it possible to compute the information rate exactly for any system. The researchers publish the results on October 26th in the journal Physical Review X.

To calculate the information rate exactly, the AMOLF researchers developed a novel simulation algorithm. It works by representing a complex physical system as an interconnected network that transmits the information via connections between its nodes. The researchers hypothesized that by looking at all the different paths the information can take through this network, it should be possible to obtain the information rate exactly. This idea, combined with advanced simulation techniques, turned out to work so well that even for systems as complex as the bacterial chemotaxis system, consisting of hundreds of chemical reactions, the information rate can now be calculated exactly.

The bacterial chemotaxis system is the information processing system that enables bacteria to swim towards food or away from toxic chemicals. Unfortunately, the amount of information that is transmitted by the bacterium cannot be directly measured experimentally: it can only be estimated from experimental data using approximations. Recently, an experimental group from Yale used this approach to estimate the information rate for the bacterial chemotaxis system. Yet, it remained unclear how accurate their estimate is.

The theory researchers of AMOLF took a different approach. Based on decades of biological experiments, an accurate model for chemotaxis had been developed. They applied their numerical technique to this model and found that the calculated rate deviated from the rate measured by the Yale group. This raised the question: what is the origin of the discrepancy between the simulations and experiments? Two answers were possible: either the chemotaxis model used by the AMOLF team was not as accurate after all, or the approximations that the Yale group had to use to arrive at their answer was wrong.

Revisiting the experimental data revealed that the existing and widely used model for the bacterial chemotaxis system was incorrect. The AMOLF researchers showed that after making a modification to fix the chemotaxis model, the calculated information rate matched the experimentally obtained rate. This analysis thus shows that the approximation used by the experimentalists to compute the information rate is, in fact, accurate for the conditions of the specific experiment. Nevertheless, this analysis also uncovers fundamental problems with the chemotaxis model developed in the past decades. Further research is required to clarify the implications of these findings for our understanding of the bacterial chemotaxis system.

The problem of computing the information rate for bacterial chemotaxis is only one example where the new simulation technique developed by the AMOLF researchers can be used to gain novel insights into physical systems. The technique is entirely general and can be applied to a wide range of systems beyond biology, ranging from optical, mechanical to quantum systems. This will deepen our understanding of information transmission via these systems, and may even lay the foundation for building novel kinds of computing devices.

Reference
Manuel Reinhardt, Gašper Tkačik, and Pieter Rein ten Wolde, Path weight sampling: Exact Monte Carlo computation of the mutual information between stochastic trajectories, Phys. Rev. X, 13, No. 4, 26 October (2023).