Unsupervised and self-supervised semantic segmentation has made significant advances in recent years in identifying and labeling regions of an image based on their appearance and context without the need for manual annotations or supervision. However, despite these advancements, the current state of the art still relies on prior knowledge, such as the number of classes present in the image. With research focusing mainly on improving the learned representations, arranging these representations into groups is left to robust but limited methods such as K-means. In this work, we propose FOCL, a computationally efficient clustering algorithm which does not require the number of clusters to be specified a priori. The idea behind FOCL is to move each point to a denser region by iteratively relocating them to the mean of their neighbors. We combine our method with a deep feature extractor to discover the underlying patterns and correlations in the representation. Our approach leverages the proven power of self-supervised learning, where the neural network learns to extract informative representations from the input images without the need for explicit human annotations. FOCL is capable of automatically discovering the number of clusters in the data, while scaling much better than Mean-shift or DBSCAN.