This paper presents an alternative adaptive parameter estimation framework for nonlinear systems with time-varying parameters. Unlike existing techniques that rely on the polynomial approximation of time-varying parameters, the proposed method can directly estimate the unknown time-varying parameters. Moreover, this paper proposes several new adaptive laws driven by the derived information of parameter estimation errors, which achieve faster convergence rate than conventional gradient descent algorithms. In particular, the exponential error convergence can be rigorously proved under the well-recognized persistent excitation condition. The robustness of the developed adaptive estimation schemes against bounded disturbances is also studied. Comparative simulation results reveal that the proposed approaches can achieve better estimation performance than several other estimation algorithms. Finally, the proposed parameter estimation methods are verified by conducting experiments based on a roto-magnet plant.