Cheng, Tin Sum Lucchi, Aurelien Kratsios, Anastasis Belius, David 2024 We derive new bounds for the condition number of kernel matrices, which we then use to enhance existing non-asymptotic test error bounds for kernel ridgeless regression (KRR) in the over-parameterized regime for a fixed input dimension. For kernels with polynomial spectral decay, we recover the bound from previous work; for exponential decay, our bound is non-trivial and novel. Our contribution is two-fold: (i) we rigorously prove the phenomena of tempered overfitting and catastrophic overfitting under the sub-Gaussian design assumption, closing an existing gap in the literature; (ii) we identify that the independence of the features plays an important role in guaranteeing tempered overfitting, raising concerns about approximating KRR generalization using the Gaussian design assumption in previous literature. application/pdf https://n2t.net/ark:/51647/srd1336035 https://sonar.unidistance.ch/global/documents/336035 https://sonar.unidistance.ch/documents/336035/files/Characterizing-Overfitting-in-Kernel-Ridgeless-Regression-Through-the-Eigenspectrum.pdf info:eu-repo/semantics/altIdentifier/doi/10.48550/ARXIV.2402.01297 info:eu-repo/semantics/altIdentifier/ark/51647/srd1336035 info:eu-repo/semantics/openAccess International Conference on Machine Learning (ICML) 2024. - 2024 Characterizing Overfitting in Kernel Ridgeless Regression Through the Eigenspectrum http://purl.org/coar/resource_type/c_5794