It was exactly like that for years, for me, I don't have a supportive family and some members, who are not what you can call exactly slim, used to drive me so nuts I would find comfort in food instead. Now I just don't give a hoot about what they think or say, even if they are saying now I must be sick to have slimmed down (and keep the weight off) and still are trying to do some body shame..... I didn't lose or keep off the weight for me but for me and ONLY for me
@Debs Working on self esteem takes time, but it works in the end, promise

Maybe people around you are not as supportive as he is ? My crazy but sweet OH has been the only one supportive around me about it and it took some time to understand that only his opinion was important, and the others (even close family) were not.