Can Finasteride cause impotence?

I just turned 24 and have been taking finasteride for hair loss since October 2009.. I never had any problems up until around March when I couldn't keep an erection and had premature ejaculation. I had also been taking Jacked3d at the time and heard that can cause some sexual side effects, so I stopped taking that and everything returned to normal. However, over the past 2 weeks, I am barely able to get an erection, ejaculating prematurely, and even when I get fully erect, it's not as hard as it used to be. Is it possible for finasteride side effects to not show up untill almost a year and a half after you start using it? And if I stop taking it, should I return to normal or is it too late?