Chromosome karyotyping analysis plays a crucial role in prenatal diagnosis for diagnosing whether a fetus has severe defects or genetic diseases. However, due to the complicated morphological characteristics of various types of chromosome clusters, chromosome instance segmentation is the most challenging stage of chromosome karyotyping analysis, leading chromosome karyotyping analysis to highly dependent on skilled clinical analysts. Since most of the chromosome instance segmentation efforts are currently devoted to segmenting chromosome instances from different types of chromosome clusters, type identification of chromosome clusters is a vital anterior task for chromosome instance segmentation. Firstly, this paper proposes an automatic approach for chromosome cluster identification using recent transfer learning techniques. The proposed framework is based on ResNeXt weakly-supervised learning (WSL) pre-trained backbone and a task-specific network header. Secondly, this paper proposes a fast training methodology that tunes our framework from coarse-to-fine gradually. Extensive evaluations on a clinical dataset consisting of 6592 clinical chromosome samples show that the proposed framework achieves 94.09% accuracy, 92.79% sensitivity, and 98.03% specificity. Such performance is superior to the best baseline model that we obtain 92.17% accuracy, 89.1% sensitivity, and 97.42% specificity. To foster research and application in the chromosome cluster type identification, we make our clinical dataset and code available via GitHub.