Skip to main content

Serious Games In The Operating Room

Da Vinci Surgical Robot--Soon to Be Kinectified

Via: The Daily of the University of Washington - UW students adapt gaming hardware for robotic surgery
January 18, 2011

A group of graduate engineering students have adapted Microsoft’s new Kinect technology for surgical robotics. The method involves using the Kinect — an array of cameras and sensors that allow videogame users to control their Xbox 360s with their bodies — to give surgeons force feedback when using tools to perform robotic surgery.

“For robotics-assisted surgeries, the surgeon has no sense of touch right now,” said Howard Chizeck, UW professor of electrical engineering. “What we’re doing is using that sense of touch to give information to the surgeon, like ‘You don’t want to go here."

Currently, surgeons commonly use robotic tools for minimally invasive surgeries. Tubes with remotely controlled surgical instruments on the ends are inserted into the patient in order to minimize scarring. Surgeons control the instruments with input devices that resemble complex joysticks, and use tiny cameras in the tubes to see inside the patient.

The problem is, however, that surgeons have no realistic way to feel what they are doing. If they move a surgical instrument into something solid, the instrument will stop but the joystick will keep moving.

Electrical engineering graduate student Fredrik Ryden solved this problem by writing code that allowed the Kinect to map and react to environments in three dimensions, and send spatial information about that environment back to the user.

This places electronic restrictions on where the tool can be moved; if the actual instrument hits a bone, the joystick that controls it stops moving. If the instrument moves along a bone, the joystick follows the same path. It is even possible to define off-limits areas to protect vital organs.

Howard’s group came up with the idea of using a “depth camera,” a sensor that detects movement in three dimensions by measuring reflecting infrared radiation to automatically define those regions. At a meeting on a Friday afternoon in December, a team member suggested using the newly released Kinect.

Before the idea to use a Kinect, a similar system would have cost around $50,000, Chizeck said.

The project is part of a larger research effort at the electrical engineering department’s BioRobotics Lab to improve surgical robotic methods. The team hopes to integrate its feedback system into a collaboration of different systems.

The team hopes to make surgical robotics reliable and practical enough for long-distance use, allowing doctors in major cities to easily perform surgeries on patients in small, isolated towns.

Ryden said that a paper will soon be published about the research. In the meantime, the sensors will need to be scaled down to a size deemed appropriate for surgical use, and the resolution of the video will need to be increased before it is usable.

Reach reporter Ryan Dunn at news at

Serious Games Bringing The Kinect Technology Into The Operating Room

Via: Winnipeg Free Press - Video Games in the OR? Doctors Say New Technology Makes Surgery More Efficient

At the same time, doctors at a Toronto hospital are banking on video game technology to save time and prevent contamination in the operating room.

A team at Sunnybrook Hospital has started using the Xbox Kinect to virtually manipulate key medical images during surgery. The doctors use hand gestures to zoom in and out of the images or freeze a particular shot without leaving the operating table.

Surgeons typically have to leave the sterile field around the patient to pull up images such as MRI or CT scans on a nearby computer. They then have to go through a meticulous cleanup before returning to the area to make sure they don't bring in any bacteria that could harm the patient. It can take up to 20 minutes to clean up each time a doctor consults an image, said Dr. Calvin Law, who helped integrate the technology into the operating room.

Those interruptions sometimes cause more than an hour's delay over the course of a surgery, said Law, a surgical oncologist with the hospital's gastrointestinal cancer team. By eliminating those delays, the hospital could save enough time to operate on more patients. What's more, it would help surgeons stay focused and decrease the risk of contamination by keeping everyone within the sterile field, he said.

With better control over the images, surgeons can be more precise, Law said. For a cancer surgeon, that could mean saving more healthy tissue when removing a tumor, he said.

The idea to bring the Kinect into the operating room came from three engineers — Jamie Tremaine, Greg Brigley and Matt Strickland.

The console is a depth camera, meaning it sees in 3-D. It then creates a digital skeleton of the person captured on camera and tracks how the skeleton moves. Those motions are translated into commands.

The engineers worked closely with surgeons at Sunnybrook to find command gestures that could be used in the operating room without compromising surgery procedures, Law said.

The system underwent extensive testing and has been used in surgery six times, Law said. There are plans to roll it out in other parts of the hospital. They're also looking into ways to use the technology for physiotherapy.