That's the problem, yes. Everything you listed could be a reason, but it could also be stress - poor soil or overcrowding can cause stress, which causes the plant to drop its flowers or young fruit and try to bloom again.
Maybe they really do need some more time. Happy gardening to you, too.