Sakura Taisen 4 was supposed to be the last title. ST5 was just a cashcow attempt to bring life back to the series.
The shortness and sloppy plot in ST4 is due to SEGA’s early abandonment of the Dreamcast. The ending to the game was actually finished first because Hiroi knew how he wanted it to end. However the actual story and gameplay in getting to the ending was still in the conceptual phase when the console support plug was pulled. SEGA rushed Red Company into completing ST4 as a “Last Hurrah” for owners of the system and fans of the game.
Which pretty much explains the failings and greatness found in ST4.
Supposedly the movie was to tie in directly with the original planned scenario in ST4, and Rachet was to make her gameplay debut as a girl to win (as the movie shows, she kinda takes a liking to Oogami). Parts of the original plot to that, were recycled in Episode 0 prequel for ST5. Ratchet of course, became a winnable girl in the actual ST5.