A Policy Gradient Reinforcement Learning (RL) technique is used to design the low level controllers that drives the joints of articulated mobile robots: A search in the controller’s parameters space. There is an unknown value function that measures the quality of the controller respect to the parameters of it. The search is orientated by the approximation of the gradient of the value function. The approximation is made by means of the robot experiences and then the behaviors emerge. This technique is employed in a structure that processes sensor information to achieve coordination. The structure is based on a modularization principle in which complex overall behavior is the result of the interaction of individual ‘simple’ components. The simple components used are standard low level controllers (PID) which output is combined, sharing information between articulations and therefore taking integrated control actions. Modularization and Learning are cognitive features, here we endow the robots with this features. Learning experiences in simulated robots are presented as demonstration.