Do I need to go exploring?

[Swote]

Hi. I'm very invested in the story, wanna finish it as fast possible without breaks. So, not going deep into sidequests. In games with ng+ I usually leave those for another playthrough. Will I be underleveled for the later story parts? Am I going to have trouble in combat?

[Croxio]

It depends on your difficulty, but if you go on easy mode it should normally be fine . And if you think you'll hit a wall it should be pretty easy to get out of it, because the levels are full of material used to help you progress.

[Ratsplat]

If you play on Story/Easy, it will be barely needed. But wouldn't hurt.

However, once you do complete the game, it is highly recommended to 100% the game before NG+

Finally, skipping ALL side areas until the end can be disappointing because so much of it will become a cakewalk. You'll be 1 shotting bosses if you have made yourself that strong by time you go back.

[Shin Happens]

You will never be underleveled for the main mission.

HOWEVER, I strongly suggest you do NOT skip the optional dungeons/levels/regions because those optional regions still have relevant story content that provide a lot more context to what is going on. The regions are not just about "fight bosses", if you really care about the story, as you say, then you will miss out on on several key scenes that greatly add to the story.

Absolutely do the Maelle sidequest you get by leveling up her relationship. Do NOT skip it. Do it before you accept the final mission in Lumiere.
Absolutely go to the Flying Manor (as the name suggests it's in the sky and you need to fly there), but beware, the fights here are somewhat harder. But you learn a few things.
Enter Renoir's atelier (access is on the ocean a bit south of the monolith), but it has to be noted that inside this level you will find probably the toughest boss in the entire game, but you also learn a few more things about what happened in the canvas world. Maybe not AS essential as the other two levels though.

So if you do not want to fight the boss in the atelier then at least do the Maelle and Flying Manor quest.

If you do it after the main story it will feel odd. And you'd wish you had known these things before, especially since they may help you decide what to do in the end.