First, let's clarify what's a window and what isn't. Please, take a look at this: https://i.imgur.com/qyBkpTs.png

Then, I'm not surprised you ask that. The game has conditioned us that everything that's usually made out of wood can also be made out of stone: doors, furniture, campfire... well, ok, not everything. But let's think about it for a second: why some things use one material while others don't? Vents, for instance, are only metal. Same with eletric equipment. I mean, in the real world, back in the 50s, there were wooden TVs, why not in the rimworld?
Well, you can certainly mod that in. But I have a theory: maybe... I don't know... maybe having different materials in the game is not an accident. Maybe it is like that to give the player something to consider when building his or her base. You know, resource managing...
Some games don't go into this kind of boring detail. In Homeworld, for instance, there are only RUs, and everything is made out of that (ok, I just gave off my age now...). But that game isn't exactly focused on resource managing... RimWorld very much is!
So here's my theory: a resource in a strategy game is a metaphor from the real world resource. It's not exaclty the same thing, it's twisted around to serve the game mechanics and entretain the player. But it's modeled after the real world thing so we can relate to it and know what to do with it.
Having said that, let's consider real world windows. Look around you: no one, at any time in history, ever used stone to make windows! People engineered openings in stone walls. But then they eventually filled it in with wooden, wickwork, paper and finally glass shutters. When I created a window for the game, I wasn't considering it just as a hole on the wall. I wanted the device that filled that hole. So that's my metaphor.
Sure, Tynan allowed for doors to be made out of stone. But dig deeper: first, there's a massive drawback, which is the opening speed. Plus, the doors are essential, It wouldn't be reasonable to ask the players to go without doors in a map without wood, right? Notice there is the real-world metaphor (door, stone) and the game needs (passages, available resources), and a compromise between the two.
"Oh no! You're breaking my immersion!!". Sorry, but that's game design for you.
So, here's my thinking on how to put windows in the game: it should be a device mounted on walls for light and venting purposes, part lamp and part vent. It should be a factor to consider on base-designing and it should not be a given, easy thing to build. And it should behave like real-world windows. Therefore... it should be wood, metal and glass!

If, you followed me so far and still believes you need stone windows, I won't argue anymore. In fact, so that we don't need to talk about this anymore, I'm going to teach you how to hack this mod and get what you want. Do this:

1. Locate the mod's defs folder on your hard drive. Typically that's C:\Program Files (x86)\Steam\steamapps\workshop\content\294100\1971860929\Defs
2. Open Things.xml in your Notepad, then find the name of the window you want to be made out of stone. Look for <stuffCategories> a few lines under that name. You'll see the material categories listed there, between <li> and </li>. All you have to do is insert a line under the last item (before </stuffCategories>) and type <li>Stony</li>. Don't touch anything else, save and exit. NOTE: that file will probably get overwritten when the mod updates, so make a backup.
3. Restart the game and enjoy!