Interested in making items for the Dota2 workshop, but not sure where to start? Welcome to my beginner/intermediate weapon creation tutorial, based on Pudges offhand weapon slot and the offhand weapon Pudges Hacker .
To give you a rundown on how this tutorial came about
I recently had one of my items, the Heavy Tenderizer for Pudge, accepted by valve and made availible in game. As im sure some of you reading this will know, this often results in a fair amount of community feedback/Q&A around item creation, texture creation and ideas.
@TehAndi contacted me around this time with some questions, we got to talking, i agreed to do a team project, which is the Pudges Hacker weapon this tutorials based around, he shared some files and it became apparent that a tutorial would be a pretty handy resource i could point people towards and hopefully continually update when new questions came in.
In saying that, this is by no means the only or best way to create items for Dota 2, just the way i happen to go about it. If you know a better more efficient process i actively encourage you to leave a comment or send me a PM, not only would i love to know myself but i can add too or amend this tutorial to include your feedback and help the community grow as a whole.
Also if at any point in this Tutorial you become stuck, feel a topic isn't covered or that i missed a step, please leave a comment at the bottom of this guide, drop me a private message, friend me Lennyagony on steam , or drop me an email at firstname.lastname@example.org, I'm eager to expand any section that may need expanding.
To get started im going to go over a bunch of less than exciting steps, software lists, directory structure and scene setup, yeah this stuff can be boring, but its best to get it out of the way early while you're still eagerly jonesing to start.
This is a list of the software i use in this guide. It's not necessarily the best software just the software i have and am most comfortable using. I encourage you to use the software you know, even if it's not covered in this tutorial as they are all incredibly similar.
Open 3DSmax and import pudge_offhand_weapon.smd , making sure to check Convert from left handed coordinate space at step four in the image below.
And you should see this in your viewport
You dont need bones in your viewport right now, so i hide them like this
1. Select by Name 2. Have only display bones checked, then select all bones 3. Right click on your selection and click Hide Selection
Now all you should have in your viewport is the cleaver, which is only in the scene for size and position reference.
At this point it can be helpful to place your reference material inside your viewport, let's do that.
First check the dimensions of your reference image in Photoshop and create a plane primitive with the same dimensions.
Next press M to bring up the material editor and
1. drag out a standard material, 2. double click the new node 3. select the tiny box nexto Diffuse, select bitmap and find your reference image. 4.check show standard map in viewport 5. right click the top of the node with the plane primitive selected and assign material to selected
You can now uniform scale the plane object to match the cleaver ref. It's also a good time to save your file and we can start pushing some polygons.
Base Mesh - Splines
This is your base mesh, which is not to be confused with your low poly game mesh. The purpose of your base mesh is to provide a base to sculpt on in Mudbox by maintaining evenly spaced quad or ngon polygons. Don't worry about your polycount at this stage as you will later optimise this mesh to create your game resolution asset.
There are a few ways i go about creating base shapes depending on the object. I'll go over a few in this tutorial starting with a spline shape for the blade and later slicing up a cube to create the handle and hilt.
With the reference scene you created above open, navigate to the
1. Create Tab 2. Click Shapes 3. Click Line 4. I have Interpolation set to 5
In you Left/Right viewport click in the viewport and start tracing your concept,
Click and Drag to create a curved line
And this is how my spline now looks,
As you can see I managed to trace the shape ok, but my vertexes are a little messy. Lets clean them up so our polygon mesh will have cleaner topology.
1. Navigate to Modify 2. chose edit spline from the modifier list 3. remove or edit the spline vertexes to look like the below image.
Also at this point i try to visualise how i want my geometry to look and edit the spline accordingly, the green lines pictured show how i expect to see the geo connect and how my final spline looks after edit.
Now let's give this spline some volume, navigate to
1. the modify tab 2. and select extrude from the modifier list
And here are the settings i used and the result.
I then navigate to the modifier list again and add Edit Poly
1. Click the polygon mode 2. Select this massive face and press the delete key on your keyboard
1. select edge mode 2. and select one of the edges on the open side of your model 3. Press the Loop button, which will select all the open edges
1. deselect the three edges circled in green by holding the left alt key and de selecting them 2. Press the Bridge tool
1. select Vertex mode 2. select target weld 3. now click and drag the single vertex circled in the below picture onto the vertex to the left of it.
And the blade should now look like this,
all you need to do now is backtrack several steps and clean up the other side of the blade and dont forget to save your file.
Refining your spline mesh
Now that you have yourself a nice shape you're going to want to refine your polygons. Mostly I want to even them out and remove some of the oddly angular sections.
To achieve this i move vertexes with Constrain to Edge turned on
Im going to skip through some of these steps and show start, middle and end shots as they are mostly repetitive vertex moving. Just select your vertex and press w to move.
Select the edges pictured below, on both sides of the blade and press connect settings
A small dialog box will appear, just slide the bottom spinner to move your newly created edges to roughly where you want the blades inner edge to appear and click the green tick.
I then adjusted the top verts by moving them with edge constraints turned on until it matches the concept, and target weld a vert at the top.
Now select the edges pictured below and press the collapse button
I also removed a couple of loops at the blades tip by selecting the edges and pressing left ctrl backspace.
At this point you should have a very practical blade base mesh that matches your concept, we will come back to this later in the tutorial to make some final edits before sculpting but for now let's move on.
Base Mesh - Box Modeling
A few things to keep in mind about the handle are, it needs to match up pretty closely with the original weapons grip in order to reduce clipping and fit nicely into the characters hand. In this instance Pudge has a pretty chunky hand, and the original cleaver had a tiny skinny handle, so im going to use some artistic licence and make the grip thicker.
I also encourage you to load up the game and checkout Pudge in the loadout, take note of how he grips his original weapon and how that translates to your creation.
This should move quicker than the blade as much of the basics covered above are the same, so moving forward make a cube with a few length segments like this
Right click the cube ->Convert To -> Convert to Editable Poly
Use the move tool and move around the verts to create a rough handle shape
Getting there but it's still pretty blocky, let's fix that by rounding out the external edges with a chamfer
1. Select all of the external edges as pictured below 2. Press the small settings box next to chamfer 3. Adjust the edge chamfer amount in the popup dialog box and press the green tick
Better, but the hilt where the blade connects the handle really needs more detail. In the reference images you can see that the steel from the blade runs all the way through the handle of the knife, with the blade, hilt and handle being one solid piece of metal, with only a small amount of material (wood in this case) added to create the grip.
I want to reproduce this look and will start by adding some loops to the hilt by either selecting the edges and using connect,
Or swift loop
And again i select verts and move them until the geo looks evenly spaced.
The front views still looking pretty square, and i may as well make use of that extra geo and put a bit of a curve in the handle. Again i just select verts and move them until i have this
Also on the side where the blade meets the hilt needs to transition from thin to thick convincingly, i start by connecting some edges
And a few more
Connect the verts
And move verts until your happy with how it looks, this is what i did.
This is a good time to do a side by side comparison of your concept, the standard weapon, your model and take a few notes, as its relatively easy to make large changes without having to do massive rework.
Base Mesh - Shape Adjustment
One thing that instantly caught my attention at this point is although the blade matches the reference, the weapon looks pretty thin and just not heavy enough for Pudge, especially in comparison to the cleaver. Dota 2's a pretty stylised game and i want to make sure i pull some of that exaggerated chunkiness into this weapon.
And for comparisons sake here's a screen grab of what you should have now and the final base mesh we are aiming for, which thankfully isn't to hard to create from what we have.
Lets reset the xform on the blade and collapse the stack, this is mostly just good practice to keep your models history in check and to avoid any unwanted mesh errors.
1. Select your blade object and go to the Utilities tab 2. Press Reset Xform 3. Press Reset Selected
1. Navigate back to the Modify Tab 2. Right click in the stack 3. press Collapse All (yes to the warning if you get one)
I want the blade to be thicker and an easy way to do that is to add a push modifier
1. Modify Tab 2. Push
and adjust the Push Value to increase the size,
Note: push works differently than scale, and is great for "inflating" objects.
add the edit poly modifier, and select all the vertexes
Scale the model, i scaled along the x axis
add a FFD 3x3x3 modifier
1. expand FFD in your stack 2. select control points 3. move the control points until you have something similar to what's pictured below
And I then just add another Edit Poly modifier onto the top of my stack, take another look at the ref and the pre edit blade. Much better, time to edit and connect up that handle.
The handle is now a little too thin for the blade and would look better shorter,
Select the verts of the bottom half and move them up
I then selected the left side and move the verts out until the handle overlaps the blade.
Its feeling pretty solid to me now, so let's connect the handle to the blade. First select the two polygons from the base of the blade and press delete.
and the two corresponding polygons from the top of the handle where the handle intersects with the blade
Move the base blade verts up to match the top of the handle
Select the edge pictured below, right click and collapse it
With the blade selected press attach and select the handle
Target weld the 5 verts on the top side of the handle to the corresponding verts on the base of the blade.
At this point I'm pretty happy with how this is looking, and in order to move onto sculpting we're going to need to even out the polygon distribution some more and setup the smoothing groups.
Base Mesh - Finalization
Initially the blade tip needs some loops removed, I'll do this by either selecting edge loops and removing them or selecting a ring of edges and collapsing them.
Add a few loops and even out the poly distribution on the body of the blade
Select this large edge ring and connect the edges
And add two loops to the handle,
Originally at this point i subdivided the mesh. Which worked out ok, but i wanted sharper edges for the blade section so i backtracked and added these three loops which help give a sharper edge on subdivision.
You may also notice that i terminated the loops at the handle, which leads to some sloppy subdivision on that small section of the weapon. I knew i wouldn't be sculpting in much detail in this area and felt it was ok in this instance to let the sloppiness slide.
I also start thinking about how im going to sculpt some of the shapes and surfaces of the model and ended up deciding i would detach the sections of mesh that will be wood. This helped me create a nice crisp seam between the metal/wood and just made sculpting this area easy.
To achieve this i select the faces that will be wood on the handle and detach them as a separate object
Change the handles name, right click the handle object and hide it.
Select the blade, select the edges pictured below and press bridge
select the edge ring pictured below and connect them
weld the six stray vertexes
select the two edges pictured below, remove them by pressing backspace and repeat the entire process for the other side of the hilt.
Right click, select hide selection, right click again, select unhide by name and unhide the handle object you hid earlier.
Now much the same process as we just went though, select the edges pictured, bridge, select edge Ring, connect
We have a small hole on the top we need to fill, but due to the triangles on either edge we can't bridge the gap. A simple solution is to select the edges pictured, hold left Shift and move along the x axis to extrude the edge, then target weld each vertex back to where you want it.
and before we forget (i just did) weld the verts on the bottom edge
Only a few more steps and the blade will be ready for sculpting, firstly you're going to need to setup your smoothing groups (hard edges in Maya), and then use NURMS subdivision and turbo smooth.
It's worth noting the smoothing groups being setup now are to aid in subdividing your mesh and will be rebuilt later, on your final game model.
Drag and select all the faces on the blade, and Clear All the smoothing groups
Now start selecting groups of faces as pictured below and setting a smoothing group to each
Unhide and set smoothing groups for the handle
Tweaks and Export Settings
Move your Objects to the centre of the grid, and move the pivot point for each object to 0,0,0 by going to the Hierarchy Tab, selecting Affect Pivot Only, type zeros into the coordinate boxes and then turn off Affect Pivot Only.
Save your file, Reset Xform and Collapse the stack for both the blade and handle. (Utilities Tab -> Reset Xform -> Reset Selected, then Modify Tab -> Right click stack -> Collapse All)
Select the blade and expand Subdivision Surface in the Modify Tab, check the settings i have pictured, in particular Use NURMS Subdivision, Use Smoothing Groups and Iterations 2.
Add the Turbosmooth modifier from the modifier list
Additionally i placed three cylinder primitives in the handle, quite simply created cylinders with a bunch of segments and changed the colours of all the items so it looks a little more like a weapon.
And we are ready to export to Mud and Sculpt!
Select both the blade and handle and goto File -> Export -> Export Selected
Export as .obj, and i use these settings
hit done and off to mudbox we go.
Sculpting - Mudbox
Let me start by saying I am by no means an advanced Mudbox user, the sculpt on this weapon happened pretty quick and im sure some of my practices are messy. I also have fewer save files, and due to that im going to attempt to deconstruct the model i have and explain the few processes i used in the creation of this asset.
Expect this part to be less in depth, and more of an overview with views towards later expansion if needed.
Something i have had to continually remind myself about while sculpting for Dota2 is these items will be baked out to 256x256 (or smaller in this case) maps, and 90% of the time viewed from the small in game perspective where only large details are visible. So try not to get carried away with the small details, keep the details large, intentional and prominent.
Let's start by importing the .obj we just exported from Max.
File ->import , select your .obj file and ok. You may see a few error messages pop up, a good example of which is pictured below. This can be problematic but for this project lets skip past them by pressing skip all.
With your weapon now loaded into the scene, save it.
And lets rebuild the subdivision levels on your objects. Go to the Select/Move tools tab, select the Objects tool, select the blade object, goto mesh, Rebuild subdivision levels and you should see a Rebuild Complete message over to the right.
Do that for all of the objects that make up your weapon
And here we have a before and after shot, the main areas i will be explaining are
Blade - roughing up the edges, creating the large cuts and creating the small surface pockmarks. Handle- Rounding out of the harder edges, counter sinking the metal bolt things and woodgrain.
I start by pressing shift D twice, which adds two more subdivision levels to the blade and bringing the polycount upto about half a million. Add a new sculpt layer by pressing the new layer button, double click the name and rename your layer to help with organisation.
Select the sculpt tool tab, select the Scrape tool and take a look at my brush settings, make sure you turn Mirror on.
I then just run around all the edges i want scuffed up with the scrape brush until I'm happy with them,
Large Surface Cuts
This area in particular was pretty messy due to the cuts not following the poly distribution and im sure could be approached with a much improved process.
On the same layer i use the standard Sculpt Tool, slowly cut into the blade with multiple strokes of varying Tool sizes, i then use the grab tool to pull the edge back.
Add another subdivision level (shift d)and create a new sculpt layer, then zoom right into the cut and clean it up as best you can with a combination of the flatten, scrape and sculpt brushes.
On the same layer i went over the entire blade with a 0.02 strength sculpt tool and added random denty noise, if you look closely you can see the round tool marks. You could just as successfully use a stencil or stamp to add noise at this stage too.
Lastly i wanted some directional detail on the sloping blade section, all i did was use the same sculpt tool in single diagonal strokes.
Select the blade mesh, press Shift F to freeze the object. If you don't do this mudbox will sculpt across all objects, which can be great in some situations. Just not this one.
To round out the handle edges, i used the flatten brush with the settings pictured. I slowly moved around all the edges and scuffed them down until they looked how i wanted.
Metal bolt things
I ramped the strength on the sculpt brush up and dug down in the handle until they all appeared, often sculpting in small circular movements to give a counter sunk bevel to the edge.
I found a nice picture of wood with heavy cracks, desaturated the image in Photoshop and imported the image into the stencil section of mudbox like so (press the tiny arrow symbol)
Subdivid the handle mesh one more time (shift d) and created a new sculpt layer. With the sculpt tool selected click on your newly imported stencil, position the stencil over the area you wish to sculpt and start sculpting
I then use the scrape tool to scrape some of the noise away from the flatter sections
And your done!
Export and get ready for baking. Select all objects, goto File -> export selected and save as a .obj file.
Now that you have a base mesh and a highpoly, it's time to create the final in game resolution model, with UVW mapping and baked base texture maps.
To get started open your Base mesh in 3dsmax, this should be the last saved file for this project before you moved onto mudbox.
From the Modify tab delete TurboSmooth off the top of the stack for all objects, turn off NURMS Subdivision and delete the metal bolt things in the handle.
Hide the handle and select the polygons (both sides) pictured below, delete them and do the same for the matching polygons on the handle.
Select the blade object and attach the handle, so that you have only one mesh.
Select all the vertexes, right click and select weld. Check my settings pictured below, if your mesh collapses reduce the threshold.
Check your polycount. To view the polycount in your viewport right click on the small + symbol in the top left of your viewport, select configure viewports, goto the Statistics Tab and check the boxes pictured in the image below.
And you should be looking at about 800 polygon Tris, not to bad but not quite the 350 we need.
Now's a good time to save your file and import your highpoly mesh. This will allow you to reduce your polycount while using your highpoly as a guide to keep everything accurate.
go to File -> Import -> Import, and select the file you exported from Mudbox, using the settings pictured below press import
The highpoly should import directly ontop of your low, if not just move either one until they overlap.
The highpoly mesh can be a fair drain on your system, if i find my frame rates dropping ill hide the highpoly, make changes and unhide to check accuracy.
To reduce the polycount to 350 ill start by selecting edge loops and removing them by pressing left ctrl and backspace. This hotkey removes the edges and verts.
At this point i notice i haven't accounted for the large cuts on the blade, they effect the silhouette so ill need to cut in some polygons and adjust the blade geometry to better reflect these changes.
Right click and collapse the edges pictured, select the one vertex and chamfer, then target weld.
Connect the two verts pictured, cut across the centre with the cut tool and then move the single vert back to create a break in the silhouette
Do the same for the other two cuts
At this point you have polygons to spare, feel free to add them where you think they are needed most, i added a few more to the bend at the top to help round out that large curve and ended up with 320. I held onto the extra 30 polygons in the event my normal map bake didn't turn out and i needed to add supporting loops, turns out i didn't need them and this is my final lowpoly mesh.
UV's and Smoothing Groups
Smoothing groups create hard edges, in game these edges create duplicated vertexes which in turn cost additional resources, similarly to using more polygons. Due to this i try to keep smoothing groups to a minimum, and use them only when i want that extra sharp line or need to improve the normal map bake (often along a uvw map seam on a hard surface). If you're not sure a good rule of thumb is to create a separate smoothing group for every UV island on your unwrap, this isn't a hard rule though so feel free to break it if it doesn't break your bake.
I'll start by UVW unwrapping the object, for this i use TeX Tools[www.renderhjs.net]. If you don't have this 3dsmax plugin, go and install it now, it's free and brilliant.
I will be mirroring my texture map, i do this to make best use of the available space and increase the texel density of my texture map. This works particularly well for a weapon as you don't see both sides at the same time and any mirroring the player may see is minimized.
To start select your model and press the Tex Tools button to bring up the textools toolbar
To complicate things a bit (booh) some of 3dsmax's UVW tools are, for lack of a better phrase, a bit♥♥♥♥♥♥unless you're working in square UV space. The way i work around this is to just pretend my space is x2 rectangular and to scale my UV's back once im done unwrapping them.
On the TexTools toolbar leave the Texture width and height square (see above), press the Edit UV button and press the Add Checker Material.
Move all your UV's out of the 0-1 UV space
Select the polygons pictured below (only one side), press Iron and then Relax
Rotate, move and scale the UV island you just created to make better use of the UV space,
Again select the polygons pictured, Iron and then Relax
Rotate, move and scale the handle UV island to fit.
You now have more than half your model unwrapped, lets mirror over the other half. Close the UV window, collapse the modifier stack, select all the polygons that are not unwrapped (covered in stretched checkers) and delete them.
With the remaining half of your object selected press the mirror button
Select any duplicated polygons, delete them, attach the other half of the object, select all of the verts, right click and weld.
To finish up your UV's you need to rescale them back to the 0-1 UV space and offset any overlapping UV's otherwise they will create errors in our bake.
With your model selected Open TexTools again, set the texture width/height to 256x128 and press Edit UV's. Where you will quickly notice your UV's are all stretched out
UPDATE: 26/04/2013 Select and scale your UV's until they fit the new space,
select all overlapping UV's,
Offset your UV's accurately by pressing the Absolute/Relative Type Ins button and inputting -1 into the V: box, once offset turn the Absolute/Relative type in tool off again by re clicking. NOTE: It's important to do this accurately so that you can reverse the process once your finished baking, which will result in mirrored UV's.
You can then just close out of the UV window, collapse the modifier stack and saved your freshly unwrapped model.
Select all of the polygons and clear all the previous smoothing groups, select the handle in its entirety and set it to Smoothing group 1
Select and set one side of the blade to smoothing group 2, select and set the other side to smoothing group 3, select only the one triangle right at the tip as pictured below and set to smoothing group 4.
Then select the top "spine" of the weapon and set to smoothing groups 2,3 and 4. This blends all three groups together and leaves you with only a hard line down the sharp end of the blade, and removed any problems i had with the sharp tip when baking out a normal map.
Save your file and let's get this ready for some Xnormal baking.
Baking - Xnormal
In order to bake in xnormal you need to make a cage, and make sure your low, cage and highpoly objects are aliened correctly before you export them.
A good start is to triangulate your lowpoly, if you let the exporter do your triangulation it can change the edge connection from export to export, which in turn changes the smoothing, which has a negative effect on your normal map.
So super simple, select all the vertexes and press connect. If you notice any triangles connecting in a way you don't like, delete the edge and connect them in a way that's more appropriate.
Now make sure your lowpoly is aligned at 0,0,0 and reset the xform. Do the same for your highpoly mesh making sure it is exactly on top of your lowpoly and also aligned to 0,0,0.
For detailed instructions on aligning your object and resetting the xform, read a few pages back under exporting to mudbox.
Duplicate your lowpoly mesh by right clicking on your lowpoly object, choosing clone from the list, make sure copy is checked and press ok.
We need to unify the normals on the cage, to do this select all the polygons and clear all the smoothing groups by pressing clear all, then set all the poly to smoothing group 1.
Now select Edit Normals from the modifier list
Select all the normals and press Unify
Add a Push modifier from the modifier list and push your cage mesh until its larger than your highpoly mesh, which was about 1.0cm in my situation.
Then reset Xforms, collapse your stack and save your file.
You should now have a lowpoly, highpoly and cage mesh in your scene, aliened to 0,0,0, with xforms reset. Select each one individually and export as .obj
goto file ->export selected, and save your file as .obj
And here are the .OBJ settings i use for each file. If everything went well you should now have three separate .obj files and we can open up some sweet sweet xnormal.
It took me a little while to warm to Xnormal but once you do it's a pretty tight little package.
Let's get right into it, Click High definition meshes, Clear all meshes then right click in the blank space, choose Add meshes from the menu and select your HIGH poly mesh.
Then click on Low definition meshes, Clear all meshes, right click in the blank space, choose Add meshes from the menu and select your LOW poly mesh.
Right click on the filename of your low poly mesh and from the menu chose Brows external cage file, and select your CAGE mesh.
Now that you have all three of your meshes loaded click Baking Options, i use the settings pictured below and bake out a batch by pressing Generate Maps
I then uncheck everything but the Normal map, reduce the size to 256x128, change the Edge padding to 4, add _small to the name and bake out my final 1-1 normal map.
And thats it, close out of Xnormal and open up Photoshop.
Textures, Mask's - Photoshop
I still find it hard to not feel like im cheating when i create a texture with current gen technology, being able to bake out so much great lighting information and just layer over a few gradients. It's at that point i remind myself the workload has just shifted from painting diffuse, to sculpting . Not a bad trade off :)
To start your going to need nDo2 (link) to generate a cavity map in this section, it's possible crazybump or the nvidia filter can do the same conversion but i haven't given them a go in a while.
You should have six large images from xnormal,
Normal Map Cavity Bent Normal Occlusion prtN prtP
Open your 1024x512 Normal map, load nDo2, and click the small convert button on the bottom of nDo2.
Then click Normal, Cavity and Active Doc
and you will have a cavity map that looks like this
Save the cavity map, and close your normal map.
At this point i create a new 1024x512 image which will be your master color map and start layering in all the baked images from xnormal, i start by creating a new group and calling it prt
Open both the prtN and prtP images, desaturate them by going to, Image -> Adjustments -> Desaturate
Copy both images into the newly created colour map, place them inside the prt Group with the prtP image set to screen mode with 85% opacity.
Open the Bent normal map, select channels, select the green channel and with only the green channel selected drag select the entire image with the rectangular marquee tool and copy.
Past the green channel you just copied into your colour map above the prt group, set it to soft light and only 9% opacity, i only needed this to be subtle this time around.
Copy the cavity map you baked in Xnormal and layer it above the BentNormal map, set to Overlay mode at 100% Opacity.
Copy the cavity map you made with nDo and layer it above the Xnormal Cavity map, set to Overlay mode at 100% Opacity.
Copy the Occlusion map and layer it above the nDo Cavity, set to multiply at 100% opacity
Duplicate the nDo cavity layer by going to Layer -> Duplicate Layer, and place it above the AO map, set to Overlay and 40% opacity
Now the important part, add a gradient MAP above the prt group by creating a new adjustment layer and selecting gradient map. Note make sure you use the gradient map and not the other gradient feature
Your image more than likely looks a bit funky right about now, fix that by clicking the gradient map bar to bring up the gradient editor. Take a look at the image below to see the gradient i used and the resulting metal effect, my advice is to fiddle with this for a bit, add nodes and move them around to see what you can come up with
Select the mask associated with the gradient map and paint black on any areas you don't want to be colored metal/grey. In this instance i masked the wood section of the handle.
Create a new gradient map above the metal gradient for the wooden handle, you will need to mask out all of the metal (inverse of the metal mask). Again please take a look at my gradient setup, this one turned out fairly crushed, ideally i should have adjusted the levels on the handle so i could spread the gradient out more - but it looks ok, will be covered in blood and pudges hand so good enough!
You now have a nice solid base for your color map, from here feel free to add whatever detail you want, i changed the values on the sharp part of the blade and some of the hard edges. To do this i created a few new layers in overlay mode with 50% opacity and painted flat white on the areas i wanted highlighted.
at this point im not happy with the side of the blade, it's a big flat area covered in mostly flat grey and not very visually interesting. To help alleviate this problem i use the gradient tool (not gradient map) to drag out a simple black and white gradient along the blade, which i then set to Multiply with 35% opacity. Mostly pretty subtle.
And that's it, a perfectly acceptable colour map.
Save the .PSD file to keep as your master and then size it down to 256x128 and save it into your final materials directory with a naming convention something like offhand_weapon_hacker_color.tga replace _hacker_ with whatever you plan to name your weapon.
Now that you have your final Normal and Colour maps, it's time to create the ever elusive mask1 and mask2.
If your having trouble understanding the mask images, i find the best way to think about them is, each image is actually four black and white images stacked one on top of the other, each performing a different task.
To begin with open the original two mask images that come with the source files, for Pudges offhand they are named pudge_clever_mask1.tga and pudge_clever_mask2.tga.
With one of the images selected click on the channels tab (nexto layers) and click through the individual R, G, B and Alpha channels to see the individual masks. As you can see each channel becomes a little clearer when viewed on its own.
I then create four separate images, one for each channel. I do this so that i can use layers and blending modes on each image, i pull layers across from my color map, often the cavity, occlusion or a desaturated color map, and adjust the levels.
When you're happy with how your four R, G, B, A images are looking, create a new image and copy each one into its associated channel .
Now save your _mask1 and _mask2 file, making sure to save them as 32 bit .tga. Tripple check your alpha channel isn't full black or full white, just put a dot somewhere on that channel otherwise the compiler will discard it.
Additionally make sure you save a version of your working files, i still end up spending allot of time reworking and re compiling my mask files to achieve a desired look. So chances are you will be coming back to these mask images.
This should mean that you now have all 4 of your materials saved out to the final material directory,
and we are ready to export and compile!
Skinning, Exporting, Compiling and Testing.
n 3DSmax open your last file, which should be the file containing your Highpoly, Lowpoly and Lowpoly Cage that you exported to Xnormal. It should also still contain the bones and cleaver you imported at the start of this tutorial (if not re import them).
Delete the Cage and Highpoly objects.
All that should remain in your scene now is, the low poly mesh you made, pudges cleaver and the imported bones.
Before moving on you need to re align the UV islands which were offset before baking(see UV section of this guide). To do this have the low poly Hacker model selected, open textools making sure the texture space is set to 256x128, press Edit UV and in the edit UV window select the offset UV islands.
Repeat the process of offsetting your UV islands by toggling on the Absolute/Relative Type ins button, inputting 1 in the V:box and toggling the Absolute/Relative Type Ins tool off again.
Now that your UV's are overlayed and back within the 0-1 UV space, Align the lowpoly mesh with the cleaver, this is important and dictates where the object appears in game.
Delete the Cleaver
Select the Lowpoly mesh, navigate to hierarchy, select Affect Pivot Only, set the pivot of the object to 0,0,0 and then turn off Affect Pivot only. This step is also important, if you dont do this your object has a good chance of floating oddly out in space beside the character.
Reset the Xform, navigate to Utilities Tab and press reset Xform and then Reset Selected.
Collapse the Modifier Stack, navigate to the Modifier Tab, right click on the stack
Edit: June 28th, Skinning your mesh, using the skin modifier.
Select your weapon model and apply the skin modifier
With your weapon still selected click Add Bones within the skin modifier. Within the Select Bones window make sure you have Display Bones turned on then select the bone weapon_lf_2 and press select.
Your weapon's now skinned to a single bone. To check this expand the skin modifier in your stack by clicking the small + icon next to the skin modifier, then select Envelope and every vertex on your weapon should turn red meaning it's being influenced by the single bone 100% of the time.
If your weapon or object's effected by more than a single bone, say a Bow that needs to flex you will need to add all of the bones associated with that weapon (check the default weapons skin modifier)and paint/adjust the skin weights accordingly.
Save your file, call it export or final or something similar.
Select the weapon model and the bone named weapon_lf_2, select File, Export, Export Selected
Save your model as a .FBX file, in the correct directory structure
I use these .FBX settings
Load Dota2, goto Store, Workshop, Wearable Item
Input all the relevant information as pictured below, and press import.
Moment of truth
And if all went well checkout your new weapon,
Chances are you may run into problems right now, maybe your object didn't import successfully? perhaps its floating? or the textures don't look anything like what you expect? don't stress it, everyone runs into problems at this point and now that you have done all the hard work its relatively simple to go back and make tweaks, just save over your files and re import your asset!
Protip: Change the output filename EVERY TIME (i just incrementally add numbers) you want to re import. As of writing, this still needs to happen or it won't update your re imported object or texture leading to much frustration.
Additionally it took me a layed back day and a half to create the weapon featured in this tutorial, it would be no problem to wrap something like this up in a day maybe less. If however your are learning the process and some of this is your first time through, chances are its going to take you a week or two, and thats normal. Learn the process, focus on quality and you will get faster.
If you made it this far please do leave me some feedback, im eager to continually improve this guide with updates based on your feedback. Anything else feel free to just shoot me a message, and be sure to follow my workshop for future updates.