We applied a Gaussian process regression framework to horizontal velocity data from ocean drifters to infer the space-time scales and total energy of rotational motions in contrasting shelf and ocean regions. A space-time covariance kernel function was chosen to represent rotational motions and its characteristic parameters were inferred directly from the data. We applied the framework to contrasting ocean and shelf environments and found consistent spatial anisotropy in the length scales of each velocity component in both regions, a feature of rotational motion dynamics. The framework resolved rotational features in a macro-tidal shelf environment beyond the limits of nadir altimetry, including a feature with a Rossby number ≈1. The framework estimated time scales in both regions to be less than 10 days, driven by a mix of temporal evolution and low frequency advection. We demonstrate the feasibility of spatiotemporal inference from sparse unstructured oceanographic observations, an approach made tractable using Vecchia's approximation to a Gaussian process, which shows potential for inference studies on large and multi-platform datasets.