In this paper we evaluate the effectiveness of Transformer models on the important task of text clustering. We present a clustering ensemble approach that harnesses all the network's layers. Numerical experiments carried out on real datasets with different Transformer models show the effectiveness of the proposed method compared to several baselines.