I guess if you phrase it as "how many calendar years were you alive for?" rather than "how many years have you been alive?" then it makes a bit more sense, but it is in many ways a less useful measure of time. It's a bit like how school years and calendar years don't line up, so you sometimes need to do a bit of a mental conversion to recall which school year you would have been in for a particular date in the past.

Semi-related, I had a buddy in high school who was born on Dec 31 and another buddy who was born in January. The January buddy was biologically a year older than the December buddy, but they both were in the same class.