Close
newsletters Newsletters
X Instagram Youtube

AI finds patterns in brain activity and turns them into pictures

Medical brain scans are displayed on a screen, with one image highlighted in red. (Adobe Stock Photo)
Photo
BigPhoto
Medical brain scans are displayed on a screen, with one image highlighted in red. (Adobe Stock Photo)
November 16, 2026 07:02 AM GMT+03:00

Researchers at the Weizmann Institute of Science in Israel have developed an artificial intelligence (AI) system called Brain-IT that recreates images people looked at using brain scan data. The authors say the tool does not read thoughts and cannot retrieve memories or imagined scenes.

The study was uploaded to the preprint database arXiv and presented at the ICLR 2026 conference, held April 23-25. The authors publicized it in September.

“We’re not reading people’s thoughts,” co-author Michal Irani, a computer scientist at the institute, told Live Science. She said the images recovered from scans depend on participants’ “full intended collaboration.”

A digital representation of the human brain is displayed against a black background. (Adobe Stock Photo)
A digital representation of the human brain is displayed against a black background. (Adobe Stock Photo)

What Brain-IT does

The system uses functional magnetic resonance imaging (fMRI), which tracks brain activity, to measure signals in the visual cortex, the part of the brain that processes visual input. Scan data is divided into voxels, small three-dimensional units of a scan.

A computer model sorts thousands of voxel measurements into 128 groups based on the type of visual information each tends to carry. The groups then exchange information, allowing the system to combine clues about a scene’s shape, color and likely objects, according to first author Roman Beliy, a doctoral candidate at the institute.

The decoder produces two kinds of predictions. One is semantic and describes what is likely to appear in a scene. For a photo of a dog standing on grass, this would include features of a dog and a grassy outdoor setting. The other is structural and describes the broad layout, such as the dog’s outline and its position in the frame.

The structural prediction gives an image generator a starting point instead of random noise. The semantic prediction guides the generator as it removes the remaining static until a picture emerges.

A digital illustration depicts data processing and information networks. (Adobe Stock Photo)
A digital illustration depicts data processing and information networks. (Adobe Stock Photo)

Built on an earlier model

Brain-IT builds on the Universal Brain Encoder, which works in the opposite direction. It starts with an image and predicts the fMRI response expected in individual voxels. Each voxel receives its own learned profile describing what it responds to in images, Beliy said. Some voxels respond to simple features such as edges, textures or locations. Others respond to categories such as faces, bodies or scenes.

Earlier image-to-fMRI systems were generally trained separately for each participant and dataset. The Universal Brain Encoder learns from data on multiple people and can transfer information across participants, datasets and MRI systems with different magnetic-field strengths, according to the study.

Brain MRI scans and diagnostic imaging results are displayed on a laptop and tablet. (Adobe Stock Photo)
Brain MRI scans and diagnostic imaging results are displayed on a laptop and tablet. (Adobe Stock Photo)

Test results

The researchers compared Brain-IT’s images with those the participants saw. The authors wrote that the system outperformed leading earlier methods on standard objective measures and produced more faithful results.

It also performed well with limited data. One hour of fMRI recordings from a new participant produced results as accurate as earlier approaches trained on 40 hours.

A realistic result is not necessarily an exact copy. The final stage uses an off-the-shelf diffusion generator trained on large numbers of images, which can smooth textures, sharpen facial features or fill in missing details.

Beliy said this “beautification” can make reconstructions look more polished and add details that do not come from the brain data. The scan constrains the image, while the AI model supplies part of its realism from patterns learned in photographs.

A medical illustration shows the human brain, eye and optic nerve pathways. (Adobe Stock Photo)
A medical illustration shows the human brain, eye and optic nerve pathways. (Adobe Stock Photo)

Why memories remain out of reach

Irani said the system relies on how the brain processes a seen image, with information moving from the eyes to the primary visual cortex and onward through areas that recognize objects and scenes. In controlled experiments, researchers know which image was shown, which gives the model a target during training and evaluation.

Memories and imagined images offer no such reference. They can be vague and shaped by networks involved in memory and thought, and they trigger only partial and weaker activity in visual regions, she said.

A medical professional examines brain scans displayed on a computer monitor. (Adobe Stock Photo)
A medical professional examines brain scans displayed on a computer monitor. (Adobe Stock Photo)

Practical limits and next steps

Reliable results currently require fMRI scanning, which is cumbersome and expensive. The team is testing electroencephalogram (EEG) data, collected through electrodes on the head, but that work is at a very preliminary stage, Irani said.

The authors say the tool could help researchers understand which visual information different brain regions represent, how regions interact and how individuals’ patterns differ.

Such tools could eventually reveal differences in visual processing that help clinicians identify neurological, neurodevelopmental and psychiatric conditions.

Irani said researchers are pursuing “numerous useful medical applications,” including using the encoder and decoder to detect and analyze multiple sclerosis (MS), post-traumatic stress disorder (PTSD), autism and psychiatric disorders.

November 16, 2026 07:02 AM GMT+03:00
More From Türkiye Today