5 o'clock in the afternoon? Or evening?

Technically, 5 pm is considered the afternoon.

Afternoon is the time of day from noon or lunchtime until evening. There is no exact definition of the time when afternoon ends and evening starts. (This is equally true for "evening" and "night".) However afternoon is often taken to start directly after noon (12:00pm). Afternoon ends and evening starts at roughly 5pm [1] or 6pm [2] or maybe just before sunset. Afternoon is the time when the Sun is descending from its zenith in the sky to somewhat before its terminus in the westerly horizon. In human life, it occupies roughly the latter half of the standard working and school day. In literal terms, it refers to a time specifically after noon.

I like evening better. Afternoon sounds so early and told me 5pm is not early. I also think 5 in the evening sounds more formal

We put "in the evening" for our 5 PM ceremony.