mudslag Posted September 23, 2011 Share Posted September 23, 2011 http://gizmodo.com/5843117/scientists-reconstruct-video-clips-from-brain-activityScientists Reconstruct Brains? Visions Into Digital Video In Historic ExperimentUC Berkeley scientists have developed a system to capture visual activity in human brains and reconstruct it as digital video clips. Eventually, this process will allow you to record and reconstruct your own dreams on a computer screen.I just can't believe this is happening for real, but according to Professor Jack Gallant?UC Berkeley neuroscientist and coauthor of the research published today in the journal Current Biology?"this is a major leap toward reconstructing internal imagery. We are opening a window into the movies in our minds."Indeed, it's mindblowing. I'm simultaneously excited and terrified. This is how it works:They used three different subjects for the experiments?incidentally, they were part of the research team because it requires to be inside a functional Magnetic Resonance Imaging system for hours at a time. They were exposed to two different groups of Hollywood movie trailers as the fMRI system recorded the brain's blood flow through their brains' visual cortex.The readings were fed into a computer program in which they were divided into three-dimensional pixels units called voxels (volumetric pixels). This process effectively decodes the brain signals generated by moving pictures, connecting the shape and motion information from the movies to specific brain actions. As the sessions progressed, the computer kept learning about how the visual activity presented on the screen corresponded to the brain activity.After recording this information, another group of clips was used to reconstruct the videos shown to the subjects. The computer analyzed 18 million seconds of random YouTube video, building a database of potential brain activity for each clip. From all these videos, the software picked the one hundred clips that caused a brain activity more similar to the ones the subject watched, combining them into one final movie. Although the resulting movie is low resolution and blurry, it clearly matched the actual clips watched by the subjects.In this other video you can see how this process worked in the three experimental targest. On the top left square you can see the movie the subjects were watching while they were in the fMRI machine. Right below you can see the movie "extracted" from their brain activity. It shows that this technique gives consistent results independently of the human or content. The three lines of clips next to the left column show the random movies that the computer program used to reconstruct the visual information.Right now, the resulting quality is not good, but the potential is enormous. Lead research author?and one of the lab test bunnies?Shinji Nishimoto thinks this is the first step to tap directly into what our brain sees and imagines: Our natural visual experience is like watching a movie. In order for this technology to have wide applicability, we must understand how the brain processes these dynamic visual experiences.The brain recorders of the futureImagine that. Capturing your visual memories, your dreams, the wild ramblings of your imagination into a video that you and others can watch with your own eyes.This is the first time in history that we have been able to decode brain activity and reconstruct motion pictures in a computer screen. The path that this research opens boggles the mind. It reminds me of Brainstorm, the cult movie in which a group of scientists lead by Christopher Walken develops a machine capable of recording the five senses of a human being and then play them back into the brain itself.This new development brings us closer to that goal which, I have no doubt, will happen at one point. Given the exponential increase in computing power and our understanding of human biology, I think this will arrive sooner than most mortals expect. Perhaps one day you would be able to go to sleep with a flexible band around your skull labeled Sony Dreamcam, wirelessly connected to your iPad 7. Pretty damn cool, maybe one day have the tech like in Strange Days. Not only recorded and see it but relive it and feel it. This video is organized as follows: the movie that each subject viewed while in the magnet is shown at upper left. Reconstructions for three subjects are shown in the three rows at bottom. All these reconstructions were obtained using only each subject's brain activity and a library of 18 million seconds of random YouTube video that did not include the movies used as stimuli. (In brief, the algorithm processes each of the 18 million clips through the brain model, and identifies the clips that would have produced brain activity as similar to the measured activity as possible. The clips used to fit the model, the clips used to test the model and the clips used to reconstruct the stimulus were entirely separate.) The reconstruction at far left is the Average High Posterior (AHP). The reconstruction in the second column is the Maximum a Posteriori (MAP). The other columns represent less likely reconstructions. The AHP is obtained by simply averaging over the 100 most likely movies in the reconstruction library. These reconstructions show that the process is very consistent, though the quality of the reconstructions does depend somewhat on the quality of brain activity data recorded from each subject. You can find more information about this work at our laboratory web site: http://gallantlab.org Link to comment Share on other sites More sharing options...
Twiddle Posted September 23, 2011 Share Posted September 23, 2011 Simply amazing! :blink: I'd surely hate to relive a nightmare though. Link to comment Share on other sites More sharing options...
DocM Posted September 23, 2011 Share Posted September 23, 2011 If you can record dreams then playback should also work. Remember the movie Strange Days? The brain recorder/player in it is closer than we thought. Ditto for Total Recall. Also has implications for vision prosthetics for the blind, the control of limb prothetics, police or other interrogations and, if made small enough, espionage. Iimagine your spouse or temporary bedmate slipping a headset on you while you sleep and getting a read on your activities. Link to comment Share on other sites More sharing options...
Twiddle Posted September 23, 2011 Share Posted September 23, 2011 If you can record dreams then playback should also work. Remember the movie Strange Days? The brain recorder/player in it is closer than we thought. Ditto for Total Recall. Also has implications for vision prosthetics for the blind, the control of limb prothetics, police or other interrogations and, if made small enough, espionage. Iimagine your spouse or temporary bedmate slipping a headset on you while you sleep and getting a read on your activities. Well I guess she will realize she needs to lose some weight and get hotter before she can get the leading role going on in my head over Emma Stone, otherwise she is settling for the cameo appearance. It will be great motivation! ....Or I'll wake up getting served with divorce papers. :p Link to comment Share on other sites More sharing options...
McCordRm Posted September 23, 2011 Share Posted September 23, 2011 In the first posted video... when the woman is on the screen talking, notice how the output image resembles a man, instead. This is going to get real cool, real quick. Science is ****ing awesome. Link to comment Share on other sites More sharing options...
mokthraka Posted September 23, 2011 Share Posted September 23, 2011 why do they use so many videos? I am lost :( Link to comment Share on other sites More sharing options...
DrJohnSmitherson Posted September 23, 2011 Share Posted September 23, 2011 Incredible.Did anyone see the episode of House where they project the dream of that person onto a computer monitor? That was so badass! Link to comment Share on other sites More sharing options...
+Xinok Subscriber² Posted September 24, 2011 Subscriber² Share Posted September 24, 2011 why do they use so many videos? I am lost :( It's all about having a large set of sample data. They record the brain activity that each video generates. Then when constructing an image from brain activity, they search for video that generated similar brain activity, and use those videos to generate an image of what the person sees. Link to comment Share on other sites More sharing options...
+Mirumir Subscriber¹ Posted September 24, 2011 Subscriber¹ Share Posted September 24, 2011 Link to comment Share on other sites More sharing options...
mudslag Posted September 24, 2011 Author Share Posted September 24, 2011 I do not remember that movie Link to comment Share on other sites More sharing options...
Fred Derf Veteran Posted September 24, 2011 Veteran Share Posted September 24, 2011 They actually only tapped into the optic centre of the brain to see what the brain was currently seeing. This process does not currently work with memories. Link to comment Share on other sites More sharing options...
Recommended Posts