Guideline On Software Using Deep Learning Technology Assisted Decision Making

1.Scope of Application
The Guideline applies to the software which uses deep learning technology assisted
decision making (including stand-alone software and software component, hereinafter
referred to as software), which are based on medical device data (medical images and
medical data generated by medical devices, hereinafter referred to as data), and are using
deep learning techniques for assisted decision making. Among them, “based on medical
device data” refers to the use of medical device data alone, or a combination of medical
device data and non-medical device data; “assisted decision making” refers to providing
medical activities to advise medical personnel to make clinical decisions.
Software that uses deep learning techniques for pre-processing (such as image quality
improvement, imaging speed improvement, image reconstruction), process optimization
(such as one-click operation), and conventional post-processing (such as image
segmentation, data measurement), etc., could refer to use the main points of this review.
Software using traditional machine learning techniques, which is not covered by this
guidance can also refer and use the main points of this review.
The main points of this review follow the relevant requirements of the guiding
principles of the “Guidelines for the Technical Review of Medical Device Software
Registration” (hereinafter referred to as the Software Guideline Principles), the “Guidelines
for the Technical Review of Medical Device Cybersecurity Registration” (hereinafter
referred to as the Cybersecurity Guidelines), the “Mobile Medical Device Registration
Technical Review” (hereinafter referred to as the Guidelines for Mobile Devices) and other
relevant guidelines.
The main points of this review only concern with technical requirements, and do not
involve legal and regulatory requirements. The software should comply with the relevant
laws and regulations on data protection and patient rights protection during the whole
product life cycle.

2. Focuses of the Main Review Points
   From the perspective of development drive factors, deep learning techniques are
   black box algorithm based on massive data with high computing power. The main points of
   this review focus on software data quality control, algorithm generalization ability and
   clinical use risk. Clinical use risk should consider the direct impact of data quality control
   and algorithm generalization capabilities, plus the indirect effects of computational
   —— 2 —
   resources (ie, operational environments) that are used by computing power.
   Risk-based full lifecycle management is the basic method of such software
   supervision. Related considerations refer to Software Guidelines, Network Security
   Guidelines, Mobile Device Guidelines, and Independent Software Addendums for Medical
   Device Manufacturing Quality Management. The following focuses on the review of
   software risk management, software design and development, software update and other
   aspects.
   Software risk management activities should be based on the intended use of the
   software (target disease, clinical use, importance, urgency), usage scenario (applicable
   population, target users, sites of use, clinical processes), core functions (handling objects,
   data compatibility and functional types) to implement and run through the software
   lifecycle process. The risk of clinical use of software mainly includes false negatives and
   false positives. False negatives are missed diagnoses, which may lead to delays in follow-up
   medical activities, especially considering the risk of delay in the diagnosis and treatment of
   rapidly progressing diseases; false positives are misdiagnosed, which may lead to
   unnecessary follow-up treatment activities. In addition to considering the risk of false
   positives and false negatives, imported software should also consider the impacts and risks
   of the differences in Chinese and foreign races, epidemiological characteristics, clinical
   diagnosis and treatment practices. Manufactures should take adequate, appropriate and
   effective risk control measures to ensure the safety and effectiveness of the software.
   The typical design and development process of such software can be divided into
   requirements analysis, data acquisition, algorithm design, verification and validation, etc.
   2.1 Requirements Analysis
   Requirements analysis should be guided by the clinical needs and using risks of the
   software, combined with the intended use, usage scenarios and core functions of the
   software, taking into account regulations, standards, users, products, data, functions,
   performance, interfaces, user interfaces, cybersecurity, alerts and other requirements; and
   focusing on data acquisition, algorithm performance, clinical use restrictions and other
   requirements.
   Data acquisition should consider the compliance and diversity of data sources,
   epidemiological characteristics of target diseases, and data quality control requirements
   (see next section for details). The data source should ensure data diversity and improve
   the generalization ability of the algorithm, such as come from the representative clinical
   institutions of multiple levels, different levels and different areas as many as possible on
   the basis of compliance to Epidemiological characteristics of target diseases include, but
   are not limited to, disease composition (eg, classification, grading, staging), population
   distribution (eg, health, patient, gender, age, occupation, region, lifestyle), statistical
   indicators (eg, morbidity, illness, disease rate, cure rate, survival rate, death rate, etc.), as
   well as the impact of target disease complications and similar diseases.
   Algorithm performance should consider false negatives and false positives (indicators,
   relationships), repeatability, reproducibility and robustness, etc.
   — 3 ——
   Clinical use restrictions should consider scenarios such situations such as clinical
   banned and used with caution.
   2.2 Data Collection
   The quality control requirements of data collection, data preprocessing, data
   annotation, data set construction and other activities should be considered to ensure data
   quality and algorithm design quality.
   2.2.1 Data Acquisition
   Data acquisition is mainly carried out by clinical institutions, and quality control
   requirements for acquisition equipment, acquisition process, and data desensitization
   should be considered.
   The quality control of the acquisition equipment should clarify the compatibility
   requirements and acquisition requirements of the acquisition equipment. The
   compatibility requirements shall be based on the data generation method (direct
   generation, indirect generation) to provide an acquisition equipment compatibility list or
   technic requirements, and specify the manufacturer, model specifications, performance
   indicators, etc. of the acquisition device. If there is no specific requirement for the
   acquisition equipment, corresponding support materials shall be provided. The acquisition
   requirements should specify the acquisition method of the acquisition device (such as
   conventional imaging, enhanced imaging), acquisition protocol (such as MRI imaging
   sequence), acquisition parameters (such as CT loading voltage, loading current, loading
   time, layer thickness), acquisition accuracy (such as resolution rate, sampling rate), etc.
   The quality control of the acquisition process should establish data acquisition
   operation specifications and clarify the requirements of the acquisition personnel and the
   requirements of the acquisition process. The acquisition personnel requirements include
   the personnel selection, training and assessment of personnel. The acquisition process
   requirements include personnel responsibilities and acquisition processes (such as
   acquisition steps and operational requirements).
   If existing historical data is used, the acquisition equipment requirements and data
   acquisition quality assessment requirements (such as personnel, methods, indicators, and
   adoption criteria) should be clearly identified.
   The collected data should be desensitized to protect patient privacy. Data
   desensitization should identify the types (static, dynamic), rules, extents, and methods of
   desensitization.
   2.2.2 Data Preprocessing
   The desensitization data is transferred from the clinical institution to the production
   enterprise to form the original database, and the data of different modalities should be
   distinguished in the original database (the same below).
   Data preprocessing should be based on the quality control requirements of data
   processing and data cleaning of the original database. Data processing should explicitly
   —— 4 —
   address methods such as filtering, enhancement, resampling, size cropping, and
   homogenization. Data cleaning should clearly define the rules and methods of cleaning.
   In data processing and cleaning, name, model specification, complete version,
   supplier, operating environment, confirmation and other requirements of the software
   tool should be clearly selected. The impact of the process and the risk of the data
   processing method on the software should be considered.
   After the data is preprocessed to form the basic database, the information such as
   sample type, sample size, and sample distribution should be clarified. The sample type can
   be divided into data sequences (composed of multiple single data, such as structural
   sequences, functional sequences and timely sequences) and data by taking the applicable
   people as the unit. The sample size should be clear about the sample size and the basis for
   determination. It is necessary to consider the impact of insufficient sample size on the
   software and its risk. Sample distribution should be based on factors such as disease
   composition, applicable population, data source institutions, acquisition equipment,
   sample types, etc., and the impact of data bias on software and its risks should be
   considered.