The rapid evolution of high-performance computing and the widespread adoption of graphics processing units (GPUs) require corresponding advances in solver design and strategies in computational geodynamics. Pseudo-transient, or dynamic-relaxation, approaches are particularly promising for multi-physics problems on structured staggered grids and are well suited to efficient parallel execution. Here, we introduce JUSTRELAX.JL, a geodynamic code written in the Julia programming language that runs seamlessly on GPUs, from a single device to distributed multi-device systems. JUSTRELAX.JL separates material-property calculations, particle-in-cell advection, and (distributed) parallelism into dedicated packages, facilitating the integration of new physical processes, numerical methods, and solver strategies while limiting modifications to the solver core. We demonstrate the capabilities of JUSTRELAX.JL using a suite of geodynamic applications, including plastic shear-band localization, thermal convection, subduction, and magmatic systems. We assess its parallel scalability on modern GPU-based supercomputers, showing efficient scaling to hundreds of GPUs and providing a foundation for future extensions in computational geodynamics.