Global warming has created a need for studies along climatic gradients to assess the effects of temperature on ecological processes. Altitudinal and latitudinal gradients are often used as such, usually in combination with air temperature data from the closest weather station recorded at 1.5–2 m above the ground. However, many ecological processes occur in, at, or right above the soil surface. To evaluate how representative the commonly used weather station data are for the microclimate relevant for soil surface biota, we compared weather station temperatures for an altitudinal (500–900 m a.s.l.) and a latitudinal gradient (49–68°N) with data obtained by temperature sensors placed right below the soil surface at five sites along these gradients. The mean annual temperatures obtained from weather stations and adjusted using a lapse rate of −5.5°C km⁻¹ were between 3.8°C lower and 1.6°C higher than those recorded by the temperature sensors at the soil surface, depending on the position along the gradients. The monthly mean temperatures were up to 10°C warmer or 5°C colder at the soil surface. The within‐site variation in accumulated temperature was as high as would be expected from a 300 m change in altitude or from a 4° change in latitude or a climate change scenario corresponding to warming of 1.6–3.8°C. Thus, these differences introduced by the decoupling are significant from a climate change perspective, and the results demonstrate the need for incorporating microclimatic variation when conducting studies along altitudinal or latitudinal gradients. We emphasize the need for using relevant temperature data in climate impact studies and further call for more studies describing the soil surface microclimate, which is crucial for much of the biota.