Researchers used an AI-driven algorithm called Kine to reconstruct this highly detailed video of the plasma jet from the active galaxy 3C 345, drawing on 116 observations gathered between 1995 and 2022. (Animation: supplied)
A new imaging method from computational imaging researchers including Aviad Levis of the University of Toronto could usher in a new era of radio astronomy by revealing the dynamics of distant astronomical objects.
The method, called Kine, was published in the journal Nature by an international team that includes Levis, an assistant professor in U of T’s Department of Computer Science.
Not all black holes are quiet. Some supermassive black holes power active galactic nuclei (or AGN), where, as Levis explains, the motion around the black hole twists the magnetic fields, “coiling them up like a spring.” That stored energy can launch enormous relativistic jets, beams of glowing plasma that move at nearly the speed of light.
By way of that jet, a black hole, although compact compared to its host galaxy, can exert an outsized influence far beyond its immediate vicinity. That makes these jets objects of astronomers’ curiosity.
A convergence of research legacies
Starting in 1995, astronomers involved with the MOJAVE monitoring program started to observe one of these black hole jets using the Very Long Baseline Array (VLBA), a network of radio dishes across North America. The resulting 116 observations occurred on an inconsistent cadence over the course of nearly 30 years. Researchers processed the data using a more traditional technique for sharpening radio images, but they could only see the brightest peaks of the jet, leaving much unobserved.
Meanwhile, a different research legacy taking shape at U of T, where University Professor Emeritus Geoffrey Hinton was developing the foundations of modern neural networks. Work by the “godfather of artificial intelligence” and 2024 Nobel laureate has found its way into a wide range of scientific applications.
These two long-term legacies of research have converged in Kine, an AI-driven algorithm built on what are called neural fields — neural networks used to represent an image or video as a continuous whole.
The Kine algorithm reconstructs continuous videos in space and time from irregularly spaced astronomical data such as the 116 separate observations of the black hole's jet. Working from so few snapshots, an accompanying Nature commentary noted, is a bit like trying to follow a film's plot from a handful of stills.
Going beyond the jet's brightest features, Kine reveals how the plasma flows throughout the entire jet. Rather than inventing detail, the technique fills in the gaps between observations in a way that stays consistent with the real data. Levis describes it as a kind of “adaptive fill-in-the-blanks.” That fuller picture let the researchers measure how the plasma actually moves across the jet, and the motion they found suggests its bright features may not be the shock waves long thought to cause them.
And Levis says the work has roots in Hinton’s early AI research at U of T. “We're actually using simple, multi-layer perceptrons, which goes back to the original work that [Hinton] was doing,” he notes.
A better look at black holes — and more
Imaging the jet is only one use for Kine. Levis notes the same algorithm is being adapted for a very different target: the glowing region right at a black hole's edge, as seen by the Event Horizon Telescope (EHT), the globe-spanning array that captured the first image of a black hole in 2019.
Because Kine reconstructs continuous motion rather than a single frame, Levis expects it to become standard practice in the field. “I foresee that this is going to be the major way that people do things in radio interferometry,” he says.
The work itself brought together researchers at the Institute of Astrophysics of Andalusia, Caltech and U of T. Levis, who is cross-appointed to the David A. Dunlap Department of Astronomy & Astrophysics, conceived the method with Caltech’s Katherine Bouman. Lead author Marianna Foschi and colleagues at the Institute of Astrophysics of Andalusia led its development.
The method’s underlying principles extend beyond astronomy. Levis says the approach draws on ideas from computer vision. Similar methods are being developed in fields ranging from microscopy of living organisms to climate science, where researchers must combine uneven observations gathered over long periods of time.
Across these many domains, computational imaging is turning scattered glimpses into coherent pictures of a changing world.