Several Jewish biblical commentators weighed in on this same question.
Rashi’s explanation—‘Because the HOBBH [Holy One Blessed Be He] was alone in the world, because the angels were not created until these second day’—seems to leave us none the wiser.
The Ramban (Nachmanides)’s interpretation is more helpful: ‘One can’t say “first day”, since the second had not yet occurred, because “the first” [implies a series that already exists], whereas “one [day]” doesn’t.’
Shadal (Samuel David Luzzatto): ‘One day, [in the sense that] the evening and the morning [together] constituted one day […] i.e. a whole day.’ To put it another way: ‘God did all this within one day—more specifically, in one evening and morning, with time left over.’
Of those three, the latter sounds most plausible. But my personal take is that the real reason is poetical: Vaihi erev, vaihi boqer, yom eḥad sounds so much more lyrical than Vaihi erev, vaihi boqer, yom rishon.
(Originally written in reply to a question at Quora.com).