Federated learning (FL) is able to utilize the computing capability and maintain the privacy of the end devices by collecting and aggregating the locally trained learning model parameters while keeping the local personal data. As the most widely-used FL framework,Jederated averaging (FedAvg) suffers an expensive communication cost especially when there are large amounts of devices involving the FL process. Moreover, when considering asynchronous FL, the slowest device becomes the bottleneck for the cask effect and determines the overall latency. In this work, we propose a communication-efficient federated learning framework with partial model aggregation (CE-FedPA) algorithm to utilize compression strategy and weighted device selection, which can significantly reduce the size of uploaded data and decrease the communication time. We perform a series of experiments on the MNIST/CIFAR-10 datasets, in both lID and non-lID data settings. We compare the communication time of different aggregation schemes, in terms of iteration rounds and target accuracy. Simulation results demonstrate that the uploading time of the proposed scheme is up to 4.3 times shorter than other existing ones. Experiments on an end - to-end FL framework also verify the communication efficiency of CE-FedPA in a real-world setting.