Adulthood is a big concept and I really don't try to convince people about what they think it is. I remember a time when I told someone I wasn't going to buy some stuff we agreed on buying together.
I explained that I had too much responsibility and he said, I don't have a wife and kids yet so I shouldn't be complaining. I told him I wasn't complaining because I am happy doing what I'm doing, it's that I don't have enough for the extra thing I wanted.
Some of us have to take care of our wives and children while also taking care of family but it's a personal struggle. Your friend was actually wrong to think that you don't have a responsibility.
They can think about what they want, just keep being yourself.