Got a Betta/Fighting Fish yesterday... now swimming back and forth?

He SHOULD be in a five gallon tank. He would be super happy, and always explor, otherwise he'll do that for a while, and then just float around.