In this manuscript, accelerated, sparse and scalable eigenstrain-based reduced order homogenization models have been developed for computationally efficient multiscale analysis of polycrystalline materials. The proposed model is based on the eigenstrain based reduced order homogenization (EHM) approach, which takes the concept of transformation field theory that pre-computes certain microscale information and considers piece-wise constant inelastic response within partitions (e.g., grains) of the microstructure for model order reduction.The acceleration is achieved by introducing sparsity into the linearized reduced order system through selectively considering the interactions between grains based on the idea of grain clustering. The proposed approach results in a hierarchy of reduced models that recovers original EHM, when full range of interactions are considered, and degrades to the Taylor model, when all grain interactions are neglected. The resulting sparse system is solved efficiently using both direct and iterative sparse solvers, both of which show significant efficiency improvements compared to the full EHM. A layer-by-layer neighbor grain clustering scheme is proposed and implemented to define ranges of grain interactions. Performance of the proposed approach is evaluated by comparison with the original full EHM and crystal plasticity finite element (CPFE) simulations.