Do I need to install the plugins separate?

[The Grand Mac]

Using OpenMW instead of just Steam itself, if it matters. I don't intend to play with mods, but any bug fixes and plugins that added content, that are not in Steam's release, I'd like to bring in.

If there are, how do I install them in the first place?

[Conkwer]

It doesn't matter if it's vanilla or OpenMW. Steam and Bethesda store versions does not include official plugins.

You can get it here:
https://en.uesp.net/wiki/Morrowind:Plugins

In vanilla you should copy them to ".\Morrowind\Data Files\" and enable them in "Morrowind Launcher" or Wrye Mash. That's it. Just like any other mod.

Same for OpenMW, I believe.

You can add plugins to openmw.cfg (at the very bottom of config), if needed. This text:
# official plugins
content=adamantiumarmor.esp
content=AreaEffectArrows.esp
content=bcsounds.esp
content=EBQ_Artifact.esp
content=entertainers.esp
content=LeFemmArmor.esp
content=master_index.esp
content=Siege at Firemoth.esp

Only "Master Index" and "Bitter Coast Sounds" plugins are worthwhile, by the way.