This! We had a very cool unit in Linguistics on this back in college, it seems the academic consensus is that the first language you learn - i.e, your native language, can stop being the primary language that you use and hence, in time, it can be forgotten.
Our professor gave us an interesting example as to why the term "native" language is no longer as relevant: her daughter, whose primary language was Romanian, had moved to Germany and met her husband there, whose primary language was German. They later lived in the US for a while, both using English as their primary language for close to a decade and then moving to Japan, where they have had their son. In essence, the kid doesn't really have a "native" language - at home, they speak English, when they visit Europe they speak Romanian or German, and everywhere else in his life he uses Japanese - which is also his primary language, as that is the one he uses most often and is most proficient in.