Cameras of the future: heart researchers create revolutionary photographic technique
14 February 2010
Scientists funded by the Biotechnology and Biological Sciences Research Council and the British Heart Foundation at the University of Oxford have developed a revolutionary way of capturing a high-resolution still image alongside very high-speed video - a new technology that is attractive for science, industry and consumer sectors alike.
By combining off-the-shelf technologies found in standard cameras and digital movie projectors they have successfully created a tool that will transform many forms of detailed scientific imaging and could provide access to high-speed video with high-resolution still images from the same camera at a price suitable for the consumer market. This could have everyday applications for everything from CCTV to sports photography and is already attracting interest from the scientific imaging sector where the ability to capture very high quality still images that correspond exactly to very high speed video is extremely desirable and currently very expensive to achieve. The technology has been patented by Isis Innovation, the University of Oxford's technology transfer office, which provided seed funding for this development and welcomes contact from industry partners to take the technology to market. The research is published today (14 February 2010) in Nature Methods.
Dr Peter Kohl and his team study the human heart using sophisticated imaging and computer technologies. They have previously created an animated model of the heart, which allows one to view the heart from all angles and look at all layers of the organ, from the largest structures right down to the cellular level. They do this by combining many different types of information about heart structure and function using powerful computers and advanced optical imaging tools. This requires a combination of speed and detail, which has been difficult to achieve using current photographic techniques.
Dr Kohl said: "Anyone who has ever tried to take photographs or video of a high-speed scene, like football or motor racing, even with a fairly decent digital SLR, will know that it's very difficult to get a sharp image because the movement causes blurring. We have the same problem in science, where we may miss really vital information like very rapid changes in intensity of light from fluorescent molecules that tell us about what is happening inside a cell. Having a massive 10 or 12 megapixel sensor, as many cameras now do, does absolutely nothing to improve this situation.
"Dr Gil Bub from my team then came up with a really great idea to bring together high-resolution still images and high-speed video footage, at the same time and on the same camera chip - 'the real motion picture'! The sort of cameras researchers would normally need to get similar high-speed footage can set you back tens of thousands of pounds, but Dr Bub's invention does so at a fraction of this cost. This will be a great tool for us and the rest of the research community and could also be used in a number of other ways that are useful to industry and consumers."
"What's new about this is that the picture and video are captured at the same time on the same sensor" said Dr Bub. "This is done by allowing the camera's pixels to act as if they were part of tens, or even hundreds of individual cameras taking pictures in rapid succession during a single normal exposure. The trick is that the pattern of pixel exposures keeps the high resolution content of the overall image, which can then be used as-is, to form a regular high-res picture, or be decoded into a high-speed movie."
The technique works by dividing all the camera's pixels into groups that are then allowed to take their part of the bigger picture in well-controlled succession, very quickly, and during the time required to take a single 'normal' snapshot. So for example, if you use 16 pixel patterns and sequentially expose each of them for one sixteenth of the time the main camera shutter remains open, there would be 16 time points at which evenly distributed parts of the image will be captured by the different pixel groups. You then have two choices: either you view all 16 groups together as your usual high-resolution still image, or you play the sixteen sub-images one after the other, to generate a high-speed movie.
This concept has attracted the attention of Cairn Research, a UK based scientific instrument manufacturer. "High speed imaging of biologically important processes is critical for many of our customers at Cairn Research," said Dr Martyn Reynolds, "Frequently there is a requirement to record events in living cells that are over in a fraction of a second, and this pushes us to the limits of existing technology. For several years we have been developing a product line for fast imaging of optical slices though cells, and we are very interested in using the processes and technology developed by the group in Oxford to extend the capabilities of our devices and the scientific benefits this could bring."
The research may soon move from the optical bench to a consumer-friendly package. Dr. Mark Pitter from the University of Nottingham is planning to compress the technology into an all-in-one sensor that could be put inside normal cameras. Dr Pitter said: "The use of a custom-built solid state sensor will allow us to design compact and simple cameras, microscopes and other optical devices that further reduce the cost and effort needed for this exciting technique. This will make it useful for a far wider range of applications, such as consumer cameras, security systems, or manufacturing control."
Dr Celia Caulcott, BBSRC Director of Innovation and Skills said: "This is a really clever, effective way of looking at real-life biological processes that started by trying to solve a research problem and is leading to whole host of opportunities. It shows that it is possible for creative solutions in bioscience tools and technologies to lead to marketable products. These researchers have been successful in making their own research tools more powerful and it is to their credit that they have also thought about the wider possibilities for their new technology."
Videos and images
The first video and image below are protected by copyright law and may be used with acknowledgement of the Nature Methods article http://dx.doi.org/10.1038/nmeth.1429
Duration: 0:00:06 (no audio). Video and audio help
The still image on the left, and the 400 fps video on the right, show a drop of milk falling into a beaker of water. Both were taken at the same time, using the same camera, and represent the same image data. The still image has a 16 fold greater spatial resolution (see swirls of milk in the beaker), and it can be decoded into the video frames played in sequence to reveal the high-speed motion content.
The video and image below are protected by copyright law and may be used with acknowledgement of Dr Gil Bub, University of Oxford.
Duration: 0:00:01 (no audio). Video and audio help
The still image on the left, and the 200 fps video on the right, show a ping-pong ball rolling past a bank note. Both were taken at the same time, using the same camera, and represent the same image data. The still image has a 16 fold greater spatial resolution (see print detail on bank note), and it can be decoded into the video frames played in sequence to reveal the high-speed motion content.
Notes to editors
The research will be published in Nature Methods at 1800hrs (GMT) on Sunday 14 February 2010: http://dx.doi.org/10.1038/nmeth.1429
For more information on the Temporal pixel multiplexing project see: http://www.physiol.ox.ac.uk/~gb1/TPMMain.html
About The British Heart Foundation (BHF)
The BHF is the nation’s heart charity, dedicated to saving lives through pioneering research, patient care, campaigning for change and by providing vital information. But we urgently need help. We rely on donations of time and money to continue our life-saving work. Because together we can beat heart disease. For more information on the BHF, visit www.bhf.org.uk/pressoffice
About the Isis Innovation
Isis Innovation is the University of Oxford's technology transfer company and manages the University's intellectual property portfolio, working with University researchers on identifying, protecting and marketing technologies through licensing, spin-out company formation and material sales. Isis files on average one new patent application each week, has concluded over 400 technology licensing agreements, and established 64 new spin-out companies from Oxford. Isis also manages Oxford University Consulting, which arranges consulting services providing clients access to the world-class expertise of the University's academics to enhance innovative capability. Last year OUC arranged over 150 consulting deals. Isis has established a separate business division, Isis Enterprise, offering consulting expertise and advice in technology transfer and open innovation to university, government and industrial clients around the world. Isis was founded in 1987 and is today one of the world's leading technology transfer and innovation management companies. www.isis-innovation.com
BBSRC is the UK funding agency for research in the life sciences. Sponsored by Government, BBSRC annually invests around £450M in a wide range of research that makes a significant contribution to the quality of life in the UK and beyond and supports a number of important industrial stakeholders, including the agriculture, food, chemical, healthcare and pharmaceutical sectors.
BBSRC provides institute strategic research grants to the following:
- The Babraham Institute
- Institute for Animal Health
- Institute for Biological, Environmental and Rural Studies (Aberystwyth University)
- Institute of Food Research
- John Innes Centre
- The Genome Analysis Centre
- The Roslin Institute (University of Edinburgh)
- Rothamsted Research
The Institutes conduct long-term, mission-oriented research using specialist facilities. They have strong interactions with industry, Government departments and other end-users of their research.
Renate Krelle, Isis Innovation
tel: 01865 280850
Jonathan Wood, University of Oxford
tel: 01865 280530
Dr Peter Kohl
tel: 01865 272533
Dr Gil Bub
tel: 01865 272533
Matt Goode, Head of External Relations
tel: 01793 413299
fax: 01793 413382
Tracey Jewitt, Media Officer
tel: 01793 414694
fax: 01793 413382