Normal map seam problem + artefacts in xnormal

[marnamai]

I have a problem with a rather ugly seam on my normal map, I am using 3ds max, mudbox and xnormal.

 

workflow

1 ) made the low poly in 3ds max (editable poly)

2 ) unwrapped it

3 ) exported as .obj from 3ds max

4 ) imported lowpoly .obj into mudbox

5 ) sculpted highpoly based on low poly

6 ) exported highpoly as .obj from mudbox

7 ) baked the normal in xnormal using the low and highpoly .obj's (has some artifacts, no idea how to fix it)

8 ) applied normal map in material in 3ds max

 

normal map seam (all channels normal)

 

 

normal map red and green flipped

 

wire on normal map

checkers

Edited April 22, 2014 by marnamai

[paranoidMonkey]

Are your seams "hard edges" (on a different smoothing group)? They must be.

For the other artifacts try using cages while baking with xnormal.

[marnamai]

the entire mesh is on a single smoothing group

How do I use a cage in xnormal?

[paranoidMonkey]

Everywhere there is a seam should be a "hard edge/split edge" or smoothing group. Try using a different smoothing group for each UV island and bake it.

You can use cages in the 3d view of xnormal. At the top would be a box with "View cage" and when you click on it should appear another "Edit cage" and a slider so you can extrude/scale the cage.

The cage should encompass the entire Highpoly model. Then save cages and bake.

 

Another thing, not related to your baking problems, you should optimize the uv space a little more imho.

 

EDIT: Don't forget to use the option "include smoothing groups" while exporting you models.

Edited April 22, 2014 by paranoidMonkey

[marnamai]

Thanks, the cage fixed the artifacts, but damn... xnormal is one cluncky piece of software ...

 

The seams are still there. :/

 

I added an extra edit poly modifier above my unwrap modifier in the stack and I assigned each island a different smoothing group, so no groups touch eachother with the same smoothing group (body = 1, legs = 2   should each island be unique as, in each leg their own unique group instead of all legs in smoothing group 2?), exported it to .obj. and rebaked it, but the issue still exists.

 

Isn't there an easy way to select the uv islands via the uvw modifier and assign smoothing groups there?

Edited April 22, 2014 by marnamai

[Pampers]

view the model in something else than 3ds max, the normalmap is most likely fine

[marnamai]

I applied a single smoothing group again after the bake, applied the normal map in a the diffuse slot and even there it is visible

 

what else can I view it in?

 

[paranoidMonkey]

Well I'm not a max user, I'm not able to answer software related questions, sorry.

But I think the 3ds max render uses +X -Y +Z so you have to invert the normal map's green channel in photoshop or bake it that way in xnormal and that should work.

Otherwise, try Marmoset Toolbag, it's a pretty cool tool for game model preview and render.

Edited April 22, 2014 by paranoidMonkey

[marnamai]

I 'll try marmoset toolbag

 

but I am pretty sure the green channel isn't the issue.

 

Here is it flipped

 

Edited April 22, 2014 by marnamai

[paranoidMonkey]

Did you tried that with the different smoothing groups bake?

[marnamai]

yes, it shows that seam in every bake I did so far, even with the channels flipped

[AyzOn]

Do you use any kind of edge splitting on these parts? (I dont know how it works in Max, I dont use it.)

You might join these parts then and bake it again, that shouldnt give shading issues as these parts are pretty smooth.

I hope that makes sense for you.^^

[paranoidMonkey]

@marnamai: meh..

Well another thing I noticed is that you're not using edge padding. Try baking it with at least 6 or 8 pixels.

[Minos]

Just a heads up, normal map display is broken in max 2013+ so NEVER use it to preview normal maps. Here's the reason why: http://www.laurenscorijn.com/future-xoliulshader-support.html

 

I'd recommend using either Marmoset or some game engine to preview it

[marnamai]

@ ayzon: I am not sure what you mean, but the model is a single mesh
@ paranoidMonkey: I had my edge padding at 2 pixels the screenshot is scaled down and it's hardly noticable, I 'll keep in mind to use more padding in the future

@ Minos: I really hope that this is the case, it's been driving me crazy and causing a headache for 2 days allready ...

But does it explain the seam when the normal is applied as a diffuse?

I 'll update once I get it to work in marmoset.

 

 

EDIT: baked an object space normal (shows no seams) converted it with handplane to a normal tangent map (shows seams)

Edited April 22, 2014 by marnamai