Knit 1, Perl 2
Becoming a surgeon is a difficult task. After 4 years at college, typically majoring in scientific specialty, there is another four years of medical school, with even more specialized study, and then a three to seven year residency program depending on the surgical specialty chosen. Typically neurosurgery requires the longest residency, roughly seven years, while ophthalmology tends to require only three. Aside from the investment in time and the value of lost wages, the cost of undergraduate college and medical school can be staggering, as seen in the table below, but the demand for surgeons continues to increase as the global population ages, making these financial barriers to entry an ever-increasing problem.
Robotic surgery, an outgrowth of minimally invasive surgery, was approved by the FDA in the US in 2000, allowing surgeons to use the systems by manipulating the device manually, initially for general laparoscopic surgery. The industry continues to grow, reaching an estimated $10.1b in 2023[1] with an increasing number of surgical procedures able to be done using these tools. The share of robotic surgery procedures has risen from 1.8% in 2012 to 15.1% in 2018[2], and certain procedures, such as hernia repair, saw growth over that same period, increasing from 0.7% to 28.8%. Robotic surgery (we know first-hand) has enabled many procedures to move from open surgery to laparoscopic, which typically means small incisions, less patient discomfort, and faster recovery, along with less bleeding and less time in the hospital.
Most hospitals have fellowships available for training in robotic surgery, along with the availability of simulators and continuing education programs that add to the understanding of the procedures by observation of more experienced users. However the learning curve is particular to the skill level of the surgeon and the difficulty of the procedures, and while simulators and visuals are important, they lack haptic feedback and real-life issues that are absolutely essential for successful robotic surgical outcomes. Actual surgical time using said tools is most important to gaining expertise, something simulators have difficulty providing. That said, with over 10 million robotic surgeries having been performed through 2021, there has been a large amount of video and kinematics data recorded during those procedures that can be used for post-operative review and training.
Most surgeons are limited in the amount of time they have available to review video of such procedures, but now that we live in the world of Ai and its ability to build multi-dimensional models from video data, researchers at Johns Hopkins and Stamford have been using this library of robotic procedures to train a robotic surgical system to perform without surgical assistance. The training procedure is called imitation learning, which allows the AI to predict actions from observations of past procedures. This type of learning system is, typically used to train service robots in home settings, however surgical procedures require more precise movements on deformable objects (skin, organs, blood vessels, etc.) at times under poor lighting, and while in theory, the videos should provide absolute mechanical information about every movement, there is a big difference between the necessary accuracy and physical mechanics of an industrial robotic arm and a surgical one.
Before AI, the idea of a surgical robot performing an autonomous procedure involved the laborious task of breaking down every movement of the procedure into 3-dimensional mechanical data (x,y,z, force, movement speed, etc.), particular to that specific procedure and was limited to very simple tasks, but it was difficult to adapt that data to what might be called normal variances. Using AI and machine learning and the AI’s ability to transform the library of video data into training data, in a way similar to how large language models transform text and images into referential data that is used to predict outcomes, the researchers say they have trained a robot to perform complex surgical tasks at the same level as human surgeons, just by watching the robotic surgeries performed by other doctors.
Here is the video of the autonomous surgical robot using the video data for refence:
https://youtu.be/c1E170Xr6BM
The researchers used the video data to train the robot to perform three fundamental tasks, manipulate a needle, lifting bodily tissue, and suturing, without programming each individual step, letting the model decide how to perform the task. According to the researchers, “the model is so good learning things we haven’t taught it. Like if it drops the needle, it will automatically pick it up and continue. This isn’t something I taught it to do.”
We understand that the idea of enabling a robotic device to pick up a needle without intervention is quite an accomplishment, however it is something that even a 1st year med student does without training, and suturing is a basic skill all surgeons learn in med school. Good surgeons are able to adapt to the circumstances they find while doing surgery, hopefully adjusting the procedure to a successful outcome regardless of the potential issues. As their experience increases, good surgeons learn how to cope with the vast number of potential problematic situations that can and do appear during surgery, not all of which relate directly to the procedure itself.
Conversely, large models draw from their training data to predict what next step in a procedure would be best, with those decisions based solely on the training data and the algorithms on which the model is based. Therefore the richness of the training data would determine how successfully the AI predicted the correct step or movement. But does the AI know that the patient is an 86-year-old male with diabetes, who was a smoker until 5 years ago, and had a single cardiac bypass procedure four years ago? Hopefully the training data includes that information, but it would also be necessary for the Ai to have been trained to understand the implications of those factors in its move-by-move decision processing. We expect that a surgeon would know the potential implications of a patient’s history and know to adjust the surgeryaccordingly, but we doubt the AI is that well equipped.
We are not putting down the idea of autonomous robotic surgery through AI, and Ai is certainly able to automate the mechanical functions of robotic devices, but good surgeons are reactive and adaptive, and use all of their senses, experience, and intelligence to make decisions during surgery, especially when performing procedures in which they specialize. We believe we are many years away from an Ai system that has the input capabilities that a human surgeon has, and while one can say that the Ai system is able to make decisions without the influence of emotion (a questionable positive), it lacks the sensory input of a human surgeon. Developing a model that has such a level of input ability and the capability to combine those varied inputs and experience information into a general model is beyond anything available today and would likely have to be so specific to the type of procedure that it would have little use as a general model. This leaves us to rely on humans to do those tasks that can be life or death, although robotics has made those tasks a bit easier. In the near-term if we could lower the cost of getting a surgeon educated and certified, that would be a real accomplishment.
[1] Straitsresearch.com – Robotic Surgery Market Size & Trends
[2] Sheetz KH, Claflin J, Dimick JB. Trends in the adoption of robotic surgery for common surgical procedures. JAMA Netw Open. 2020;3(1):e1918911
Most hospitals have fellowships available for training in robotic surgery, along with the availability of simulators and continuing education programs that add to the understanding of the procedures by observation of more experienced users. However the learning curve is particular to the skill level of the surgeon and the difficulty of the procedures, and while simulators and visuals are important, they lack haptic feedback and real-life issues that are absolutely essential for successful robotic surgical outcomes. Actual surgical time using said tools is most important to gaining expertise, something simulators have difficulty providing. That said, with over 10 million robotic surgeries having been performed through 2021, there has been a large amount of video and kinematics data recorded during those procedures that can be used for post-operative review and training.
Most surgeons are limited in the amount of time they have available to review video of such procedures, but now that we live in the world of Ai and its ability to build multi-dimensional models from video data, researchers at Johns Hopkins and Stamford have been using this library of robotic procedures to train a robotic surgical system to perform without surgical assistance. The training procedure is called imitation learning, which allows the AI to predict actions from observations of past procedures. This type of learning system is, typically used to train service robots in home settings, however surgical procedures require more precise movements on deformable objects (skin, organs, blood vessels, etc.) at times under poor lighting, and while in theory, the videos should provide absolute mechanical information about every movement, there is a big difference between the necessary accuracy and physical mechanics of an industrial robotic arm and a surgical one.
Before AI, the idea of a surgical robot performing an autonomous procedure involved the laborious task of breaking down every movement of the procedure into 3-dimensional mechanical data (x,y,z, force, movement speed, etc.), particular to that specific procedure and was limited to very simple tasks, but it was difficult to adapt that data to what might be called normal variances. Using AI and machine learning and the AI’s ability to transform the library of video data into training data, in a way similar to how large language models transform text and images into referential data that is used to predict outcomes, the researchers say they have trained a robot to perform complex surgical tasks at the same level as human surgeons, just by watching the robotic surgeries performed by other doctors.
Here is the video of the autonomous surgical robot using the video data for refence:
https://youtu.be/c1E170Xr6BM
The researchers used the video data to train the robot to perform three fundamental tasks, manipulate a needle, lifting bodily tissue, and suturing, without programming each individual step, letting the model decide how to perform the task. According to the researchers, “the model is so good learning things we haven’t taught it. Like if it drops the needle, it will automatically pick it up and continue. This isn’t something I taught it to do.”
We understand that the idea of enabling a robotic device to pick up a needle without intervention is quite an accomplishment, however it is something that even a 1st year med student does without training, and suturing is a basic skill all surgeons learn in med school. Good surgeons are able to adapt to the circumstances they find while doing surgery, hopefully adjusting the procedure to a successful outcome regardless of the potential issues. As their experience increases, good surgeons learn how to cope with the vast number of potential problematic situations that can and do appear during surgery, not all of which relate directly to the procedure itself.
Conversely, large models draw from their training data to predict what next step in a procedure would be best, with those decisions based solely on the training data and the algorithms on which the model is based. Therefore the richness of the training data would determine how successfully the AI predicted the correct step or movement. But does the AI know that the patient is an 86-year-old male with diabetes, who was a smoker until 5 years ago, and had a single cardiac bypass procedure four years ago? Hopefully the training data includes that information, but it would also be necessary for the Ai to have been trained to understand the implications of those factors in its move-by-move decision processing. We expect that a surgeon would know the potential implications of a patient’s history and know to adjust the surgeryaccordingly, but we doubt the AI is that well equipped.
We are not putting down the idea of autonomous robotic surgery through AI, and Ai is certainly able to automate the mechanical functions of robotic devices, but good surgeons are reactive and adaptive, and use all of their senses, experience, and intelligence to make decisions during surgery, especially when performing procedures in which they specialize. We believe we are many years away from an Ai system that has the input capabilities that a human surgeon has, and while one can say that the Ai system is able to make decisions without the influence of emotion (a questionable positive), it lacks the sensory input of a human surgeon. Developing a model that has such a level of input ability and the capability to combine those varied inputs and experience information into a general model is beyond anything available today and would likely have to be so specific to the type of procedure that it would have little use as a general model. This leaves us to rely on humans to do those tasks that can be life or death, although robotics has made those tasks a bit easier. In the near-term if we could lower the cost of getting a surgeon educated and certified, that would be a real accomplishment.
[1] Straitsresearch.com – Robotic Surgery Market Size & Trends
[2] Sheetz KH, Claflin J, Dimick JB. Trends in the adoption of robotic surgery for common surgical procedures. JAMA Netw Open. 2020;3(1):e1918911
RSS Feed