The famous prime orbit theorem counts the number of closed geodesics on compact negatively curved Riemannian manifolds. On top of that, we know closed geodesics are equidistributed with respect to the Liouville measure (or the Bowen-Margulis measure). In this talk, we will discuss how to generalize these results to non-compact and non-Riemannian settings. The main tool is the thermodynamic formalism of countable state Markov shifts and the renewal theorem. If time permits, we will discuss applications in higher Teichmüller theory. This is a joint work with Harry Bray, Dick Canary, and Giuseppe Martone.