Ann, I'm sure you're not letting this lovely little cat starve, but if you are unable to adopt it, you could get in touch with your nearest branch of catsprotection.co.uk to see if they can rehome it.
Oh, I see. Well, it's good that you're there for him and that he can come in if he wants!
My Mickey was like that for a long time and still hates being locked in (on Guy Fawkes NIght) and is scared of visitors, but after years of perseverance, he's now a happy pet most of the time.