Ultra-short-term photovoltaic (PV) power forecasting can support the real-time dispatching of the power grid. However, PV power has great fluctuations due to various meteorological factors, which increase energy prices and cause difficulties in managing the grid. This article proposes an ultra-short-term PV power forecasting model based on the optimal frequency-domain decomposition and deep learning. First, the optimal frequency demarcation points for decomposition components are obtained through frequency-domain analysis. Then, the PV power is decomposed into the low-frequency and high-frequency components, which supports the rationality of decomposition results and solves the problem that the current decomposition model only uses the direct decomposition method and the decomposition components are not physical. Then, a convolutional neural network (CNN) is used to forecast the low-frequency and high-frequency components, and the final forecasting result is obtained by addition reconstruction. Based on the actual PV data in heavy rain days, the mean absolute percentage error (MAPE) of the proposed forecasting model is decreased by 52.97%, 64.07%, and 31.21%, compared with discrete wavelet transform, variational mode decomposition, and direct prediction models. In addition, compared with recurrent neural network and long-short-term memory model, the MAPE of the CNN forecasting model is decreased by 23.64% and 46.22%, and the training efficiency of the CNN forecasting model is improved by 85.63% and 87.68%. The results fully show that the proposed model in this article can improve both forecasting accuracy and time efficiency significantly.