Defect centers in two-dimensional hexagonal boron nitride (hBN) are drawing attention as single photon emitters with high photo-stability at room temperature. With their ultra-high photon-stability, hBN single photon emitters are promising for new applications in quantum technologies and for two-dimensional material based optoelectronics. Here, we control the emission rate of hBN-defects by coupling these to resonant plasmonic nanocavities. By deterministic control of the antenna we acquire high-resolution emission maps of the single hBN-defects. Using time-gating, we can discriminate the hBN defect emission from the antenna luminescence. We observe sharp dips (40 nm FWHM) in emission, together with a reduction with the luminescence lifetime. Comparing with FDTD simulations we conclude that both radiative and non-radiative rates are enhanced, which effectively reduce the quantum efficiency. Also, the large refractive index of the hBN largely screens off the local antenna field enhancement. Finally, based on the insight gained, we propose a close-contact design for an order of magnitude brighter hBN single photon emission.