Many people go to college for different reasons. For me it is all about the academics and social life comes afterward. Yes I think you should politely decline this invitation. It would be helpful for your sweet mates to have contact information for your host family.