Surrogate models are becoming increasingly popular for storm surge predictions. Using existing databases of storm simulations, developed typically during regional flood studies, these models provide fast-to-compute, data-driven approximations quantifying the expected storm surge for any new storm (not included in the training database). This paper considers the development of such a surrogate model for Delaware Bay, using a database of 156 simulations driven by synthetic tropical cyclones and offering predictions for a grid that includes close to 300,000 computational nodes within the geographical domain of interest. Kriging (Gaussian Process regression) is adopted as the surrogate modeling technique, and various relevant advancements are established. The appropriate parameterization of the synthetic storm database is examined. For this, instead of the storm features at landfall, the features when the storm is at closest distance to some representative point of the domain of interest are investigated as an alternative parametrization, and are found to produce a better surrogate. For nodes that remained dry for some of the database storms, imputation of the surge using a weighted k nearest neighbor (kNN) interpolation is considered to fill in the missing data. The use of a secondary, classification surrogate model, combining logistic principal component analysis and Kriging, is examined to address instances for which the imputed surge leads to misclassification of the node condition. Finally, concerns related to overfitting for the surrogate model are discussed, stemming from the small size of the available database. These concerns extend to both the calibration of the surrogate model hyper-parameters, as well as to the validation approaches adopted. During this process, the benefits from the use of principal component analysis as a dimensionality reduction technique, and the appropriate transformation and scaling of the surge output are examined in detail.