We demonstrate second and third harmonic generation from a GaAs substrate, far from the phase matching and well-below the absorption edge, in both transmission and reflection geometries. The pump is tuned in the transparency range, while the SH and the TH signals are tuned in the opaque spectral range of GaAs. We find that the polarization of the generated signals is sensitive to the polarization of the pump. The experimental results are contrasted with numerical simulations that account for both surface and bulk contributions and show that the bulk-generated SH components are less intense than surface-generated SH signals.