Secure communications have become a requirement for virtually all kind of applications. Currently, two distant parties can generate shared random secret keys by using public key cryptography. However, quantum computing represents one of the greatest threats for the finite complexity of the mathematics behind public key cryptography. In contrast, Quantum Key Distribution (QKD) relies on properties of quantum mechanics, which enables eavesdropping detection and guarantees the security of the key. Among QKD systems, polarization encoded QKD has been successfully tested in laboratory experiments and recently demonstrated in closed environments. The main drawback of QKD is its high cost, which comes, among others, from: i) the requirements for the quantum transmitters and receivers; and ii) the need of carefully selecting the fibers supporting the quantum channel to minimize the environmental effects that could dramatically change the polarization state of photons. In this paper, we propose a Machine Learning (ML) -based polarization tracking and compensation that is able to keep shared secret key exchange to high rates even under large fiber stressing events. Exhaustive results using both synthetic and experimental data show remarkable performance, which can simplify the design of both quantum transmitter and receiver, as well as enable the use of aerial optical cables, thus reducing total QKD system cost.