In response to the growing demand for deploying deep learning models on resource-constrained devices, my research develops innovative quantization techniques to optimize deep neural networks without sacrificing performance. This work introduces a novel method that leverages a multi-layer perceptron to determine optimal layer-wise bit-widths, using the Kullback-Leibler divergence between quantized and full-precision outputs as a metric. By employing Monte Carlo sampling to navigate the exponential search space and incorporating a penalty term to minimize network size, the approach significantly reduces computational overhead. Demonstrated on architectures such as VGG16, ResNet50, and GoogLeNet, this method achieves up to 6x, 4x, and 4x compression respectively on the ImageNet dataset, maintaining accuracy while enhancing efficiency and paving the way for scalable, edge-deployable deep learning solutions.
My research delves into the interplay between loss curvature and model privacy in deep neural networks. By examining the trace of the Hessian of the loss with respect to the input—termed input loss curvature—I explore how this curvature varies between training and testing datasets and its implications for train-test distinguishability. Building on this analysis, I have developed a theoretical framework that establishes an upper bound on distinguishability in relation to privacy constraints and the training set size. This framework underpins a novel black-box membership inference attack that leverages input loss curvature, demonstrating superior effectiveness on large-scale datasets such as CIFAR10, CIFAR100, and ImageNet. My work not only deepens our understanding of network behavior but also informs the development of more robust privacy-preserving techniques in machine learning.
My research investigates the intrinsic connections between memorization, generalization, and privacy in deep neural networks. In particular, I examine how input loss curvature—defined as the trace of the loss Hessian with respect to the inputs—can serve as a practical and efficient proxy for measuring memorization. By developing a theoretical framework, I derive upper bounds on memorization that link differential privacy parameters with input loss curvature. This work not only provides new theoretical insights into the behavior of deep learning models but also demonstrates strong empirical correlations on datasets such as CIFAR and ImageNet, offering a unified view of how these networks balance memorization and privacy.