Plans for a Buddhist temple in Cambridge are under threat after leading monks have been forced out of the country.