Medical Robots Dramatically Reduce Infection Rates! Proposal Of A Cooperative Control Method For The Use Of Robots In Clinical Settings!
3 main points
✔️ Attention has been focused on the implementation of robots in tasks that can be performed in a specific pattern
✔️ This study aims at a robot cooperation model for the introduction of medical robots to provide medical services in people directly exposed to COVID-19
✔️ The results suggest that Q-learning is effective for temporal and spatial complexity
A reinforcement learning based algorithm for personalization of digital, just-in-time, adaptive interventionsPrevention of Covid-19 affected patient using multi robot cooperation and Q-learning approach: a solution
written by
(Submitted on May 2021)
Comments: Artif Intell Med
The images used in this article are from the paper, the introductory slides, or were created based on them.
background
Is it possible to introduce medical robots in high infection risk sites?
In this study, we aim to construct an algorithm to derive optimal cooperative policies based on reinforcement learning, which will facilitate the introduction of medical robots in the front line of infectious diseases.
COVID-19, which continues to rage, is transmitted through droplets of coughs and sneezes from infected people and contaminated surfaces, spreading from one infected person to an average of three to ten people; healthcare workers, on the other hand, have more contact with patients and are at dramatically higher risk of infection, so to prevent infection among healthcare workers Robots are being considered for implementation: specifically, focused implementation has begun in patient care tasks - feeding, medicating, and watering patients. On the other hand, there are limited reports describing coordination among these robots - the lack of adaptive coordination among them increases the likelihood of malfunction and disruption, and makes it more difficult to achieve effective treatment.
We propose a reinforcement learning-based cooperative algorithm between robots to control the rate of spread of viruses by reducing the exposure of healthcare workers to patients and infections: specifically, we construct deterministic action selection in a controlled environment using Q-learning and show that each step of the Performance is measured and trained using an objective function based on the algorithm and parameters used. The robot in this study was envisioned to operate as follows; transporting patients to target beds: moving patients to available beds from the time the ambulance arrives; providing medication and meals: providing medication and meals to patients on time; on-call service provider: providing services such as water distribution to patients on request; and Emergency controller to call a doctor
Based on this process, this paper proposes a novel approach to (1) reduce the exposure potential of medical personnel, (2) reduce the number of medical personnel by deploying medical robots, (3) classify groups of medical robots based on their task assignments, (4) apply reinforcement learning approaches to robot path finding, and (5) compute collision avoidance paths and co The main topics of this paper are cooperation among robots.
What is COVID-19?
Here, we briefly describe COVID-19, which is the subject of our analysis.
COVID-19 was discovered in Wuhan, China, in 2019 and then spread globally, causing a pandemic - as of 2022, it has previously raged, including new variants. The virus can develop symptoms four or five days after infection - as long as two weeks later. The main symptoms are: fever; cough; breathlessness; lethargy; loss of smell and taste. Elderly people and patients with underlying diseases such as heart disease and diabetes are more likely to develop severe pneumonia, and other generations have also reported respiratory symptoms, high fever, diarrhea, and taste disturbance. Genetic sequencing analysis has reported that the virus is similar to the coronaviruses of bats and civets, suggesting that these may have undergone genetic recombination. The number of cases is reported to be about 1,000 in Japan. It is spread from human to human via cough and droplets, mainly by airborne transmission. Currently, the development of a highly effective vaccine is underway to prevent the spread of the disease.
technique
Problem formulation
This section describes the formulation of the problem to be solved in this study.
In this evaluation, several robots working in the COVID-19 medical environment are assumed and formulated - see figure below.
These robots are divided into two groups based on their assigned functions: the
(1) Patient transfer robots: Two robots will be assigned for the purpose of transferring new patients from the entrance to the target empty beds in the COVID-19 hall. They are equipped with image sensors and stand by near the entrance. Once the arrival of the patient is confirmed, they perform the functions of holding, positioning, moving, and returning the patient to the target with the patient on the stretcher. The robot must meet a distance constraint, indicated by the length of the stretcher - if the distance constraint is greater than that, it will have problems moving the patient from one position to another. Here, Q-learning is used to calculate the next position at each step towards the target while avoiding collisions.
(2) Service Provider Robot: Provides services such as medicine, food and water to patients and monitors their health status. These are assumed to be in a room. Equipped with time and temperature sensors, the patient's medication and food times are set by the robot - the functions describing this task are denoted as PRO-MED (R, i, t) and PRO-SER (R, i, t), respectively. At each robot, the information is stored in an array and the entries represent the patient's timing. The temperature sensor senses the patient's body temperature and transmits the data to the relevant medical personnel - or, in case of an emergency, it can make a video call to the medical personnel so that they can take the necessary actions. Some of the robots are also equipped with image and temperature sensors to monitor the health status of the patient while listening to their needs.
To avoid collisions between robots, we have also built an algorithm to derive a suitable path - from the position assigned to the robot, we calculate the optimal path to reach the target patient. These derivations are based on the difference between the desired shortest path 𝐷A and the actual path 𝐷C, as well as the objective function. The paths are computed as in Equations 1 and 2.
Similarly, the objective function of the robot is also calculated to minimize the delay cost incurred by the turns and rotations performed by the robot - see equation below.
operating model
In this section, we describe the motion model that the robot performs.
For the construction of the theoretical model, the patient hall of COVID-19 is defined as an area of approximately 400 × 500 pixels, in which the bed and the robot are arranged in a grid. The robot is equipped with 6-DOF wheels. For the calculation of the next state, the work procedure is divided into two cases by robot groups: the
Case 1: Applies to the work procedure of the robot in Group 1. Assume that both robots carry the COVID-19 patient to an empty bed called TARGET in the figure. The robot performs the following steps so that the distance between the robots is equal to the length of the stretcher in one move.
Case 2: Group 2 robot target, a single robot that can move in all directions, similar to the group1 robot. Based on the distance and direction of the target, the robot selects the next step.
The algorithm for these robots begins with initialization of the robot's position and group. Depending on the initialized groups, the tasks are different: group 1 finds the cooperating robots in the group, senses the arrival of the patient, and then moves towards the target where the patient is - if there is an obstacle, it updates the Q table and searches for the next step; group 2 identifies the target patient and calls the FIND function - this function returns the next position to serve, avoiding collisions. Similarly, the robot detects the patient's request and provides the service. The robot is also responsible for monitoring the patient's health status - if the body temperature deviates from the threshold, it calls the doctor urgently.
result
This section describes the evaluation.
risk analysis
We used the dataset from COVID-19 to assess the risk of introducing robots. Viral pandemics were assessed using R naught values - the rate of spread from one person to another.
These data sets also show that those working on the front lines of COVID-19 are at high risk of infection - see figure below; on the other hand, the introduction of medical robotics for this population can reduce the frequency of contact and reduce the infection rate among healthcare workers to the conventional 2%. The results of this study will help to reduce the number of patients.
comparative analysis
A comparison is made between the proposed method and existing methods. Here, we analyze the time space used and the effectiveness of the methods - the complexity of the space; the computational complexity
Complexity of space
The proposed algorithm requires a space to store the Q table. In the evaluation, the set COVID-19 holes are represented by a matrix of N - the number of states - × M - the number of actions associated with each state, so an N × M matrix is required; therefore, the space The complexity is O(MN), and the maximum Q value needs to be determined for the next state identification. On the other hand, the spatial complexity can be reduced to O(N) by setting up a storage area to store Q values and locking variables.
time complexity
Q-learning requires access to the Q-table for each action and M - 1 comparisons - so the number of accesses in N robots is N × (M - 1) and the time complexity is O (MN); on the other hand, assuming implementation in a real clinical setting, the robot's Since the action set is restricted to 8 actions, the constant term M can be neglected and the complexity is suppressed to O (N).
consideration
In this study, we proposed a collaboration algorithm between robots based on reinforcement learning for the purpose of promoting support for medical professionals.
To prevent the spread of COVID-19, preventive measures are needed for healthcare workers and other frontline employers - attention has focused on the introduction of robots to reduce the risk of infection to these individuals. In this paper, we develop a model to derive optimal deployment and deployment policies for medical robots that transport patients, provide food and medicine, and respond to medical emergencies: specifically, we use reinforcement learning - Q-learning - to develop a model that can be used to determine the optimal deployment of medical The optimal policy for robots to cooperate with each other is derived. It was confirmed that the model has a high potential to reduce the mortality rate of healthcare workers to 2%.
One of the strengths of this research is the proposed derivation algorithm that focuses on the cooperative nature of medical robots. The problem of how to prevent the spread of COVID-19 infection in people at high risk of infection, which is still rampant, is urgently needed. In these settings, specific patterns of tasks - patient transport and service delivery - are expected to be performed by medical robots, and these patterns can be replaced by medical robots to reduce the risk of infection and cost: these Since the task requires the introduction of multiple robots, the purpose of this research, which focuses on cooperative control, is considered to be novel. Another advantage is the simplicity of the algorithm used. The complexity of the learning process and the high cost of learning have been pointed out as problems in the introduction of reinforcement learning to clinical practice. The more complex the algorithm, the more difficult it is to implement. Another advantage of this method is that it can be used to improve the quality of the data.
On the other hand, the issues are the cost of introducing the robot to clinical sites and sharing knowledge with site personnel: Specifically, the initial investment in introducing the robot - i.e., the purchase of hardware and software - is required, and the robot cannot be introduced in some regions. The possibility of this is a consideration. As a solution to this problem, it is possible to reconstruct the target of the algorithm used in this study not as a robot but as a person working in the field, and to derive a model for deriving an optimal deployment policy that minimizes the risk of infection. In addition, it is assumed that the knowledge of the on-site workers needs to be supplemented for the effective use of the medical robots. For these issues, manuals on the use of robots and cooperation with experts are considered as possible solutions.
Categories related to this article
