Economists worry that a sharp drop in immigration could weaken economic growth as the current workforce ages and starts to retire.