光华讲坛——社会名流与企业家论坛第 期

主 题：Significance tests of feature relevance for a black-box learner

黑匣子learner特征相关性的显著性检验

主讲人：香港中文大学戴奔博士

主持人：统计学院林华珍教授

时间：2024年3月8日（周五）下午15:00-16:00

举办地点：柳林校区弘远楼408会议室

主办单位：统计研究中心和统计学院 科研处

主讲人简介：

戴奔是香港中文大学统计系的助理教授。他的主要研究兴趣包括基于理论的机器学习方法、机器学习的理论基础、黑匣子显著性检验、统计计算和软件开发。

内容简介：

An exciting recent development is the uptake of deep neural networks in many scientific fields, where the main objective is outcome prediction with a black-box nature. Significance testing is promising to address the black-box issue and explore novel scientific insights and interpretations of the decision-making process based on a deep learning model. However, testing for a neural network poses a challenge because of its black-box nature and unknown limiting distributions of parameter estimates while existing methods require strong assumptions or excessive computation. In this article, we derive one-split and two-split tests relaxing the assumptions and computational complexity of existing black-box tests and extending to examine the significance of a collection of features of interest in a dataset of possibly a complex type, such as an image. The one-split test estimates and evaluates a black-box model based on estimation and inference subsets through sample splitting and data perturbation. The two-split test further splits the inference subset into two but requires no perturbation. Also, we develop their combined versions by aggregating the p -values based on repeated sample splitting. By deflating the bias-sd-ratio, we establish asymptotic null distributions of the test statistics and the consistency in terms of Type 2 error. Numerically, we demonstrate the utility of the proposed tests on seven simulated examples and six real datasets. Accompanying this article is our python library dnn-inference (https://dnn-inference.readthedocs.io/en/latest/) that implements the proposed tests.

最近一个令人兴奋的发展是深度神经网络在许多科学领域的应用，其主要目标是具有黑匣子性质的结果预测。显著性检验有望解决黑匣子问题，并探索基于深度学习模型的决策过程的新颖科学见解和解释。然而，神经网络的测试面临挑战，因为它的黑匣子性质和参数估计的未知极限分布，而现有的方法需要强大的假设或过多的计算。在本文中，主讲人推导了单分裂和双分裂测试，放宽了现有黑匣子测试的假设和计算复杂性，并扩展到检查可能是复杂类型的数据集(如图像)中感兴趣的特征集合的意义。单分裂检验通过样本分裂和数据扰动对基于估计和推理子集的黑匣子模型进行估计和评估。双分裂检验进一步将推理子集分成两个，但不需要扰动。此外，主讲人通过基于重复样本分割的p值聚合来开发它们的组合版本。通过缩小偏sd比，主讲人建立了检验统计量的渐近零分布和2型误差的一致性。在数值上，我们在7个模拟实例和6个真实数据集上证明了所提出的测试的有效性。本文附带了我们的python库dnn-inference (https://dnn-inference.readthedocs.io/en/latest/)，它实现了提出的测试。