On the carriers I've been on, the only reason officers might not leave until later is because they had work to do. Officially, they have the same duty rotation as us lowly blue shirts, and very rarely have I seen them leave much later than the rest of us. On pull in days, expect to lose about 3-4 hours of liberty getting plants set up while the rest of the ship leaves.
On my last ship, we changed the duty section turn over times to a little before noon, that way guys with overnight liberty could actually enjoy it.