First, I assume you know that the two definitions yield objects which are not just isomorphic, but identical: from the 'left' definition it follows that the 'left' identity is also a 'right' identity and ergo it is 'the' identity, and the 'left' inverse is also a 'right' inverse and ergo 'the' inverse; and vice versa follows for the 'right' definition.
So I would say that the formulation of the axioms has little to do with Burnside's lemma per se; and instead is intended to provide a minimum set of requirements: only one side, left or right, is required to deduce that the resulting entities (identity, inverse) are actually /both-sided/. This is in contrast to other types of structures; e.g., modules, where a left module is not necessarily a right module.
Also, historically, the notation for the application of a group operation has varied between right to left and left to right over time. For example in W. R. Scott's "Group Theory" (1964), assuming x in S and g,h in G where G acts on S, the author writes "xhg" for what would now more typically would be written as "ghx". This has other terminological effects such as the precise meaning of "left coset" or "right coset", but of course the essential theory is the same.