Dirty Bomb - tools

[blackdog]

Stumbled on this by related-videos on YT

Do you do a lot of profiling on the (multiplayer) games you are working on?

[Buddy]

This thread will be picked up by the CIA...

[selmitto]

Veeeeery interesting! Thank you for sharing. Awesome to know that Dave Johnston is involved.

[marks]

Hasn't DaveJ been at SD for quiiiite a long time now ..?

[AlexM]

That sounds amazing. I'm suddenly super interested in this game.

[blackdog]

On topic please... ghghgh

No seriously, is this such secretive stuff that anybody cannot talk about...?

I mean, I'm not asking details, it was just to understand how widespread this practice is.

The particular video show such beautiful tools that makes me wonder if is not viral marketing, and they are going to start selling those as a solution for everybody who is interested, you know, integrate our APIs and get this beautiful reports. (Unless that was just all fake.)

[Algor]

It has been around for a while. TF2 and other Valve games use this. There used to be easy ways to view it on the official CS:S and TF2 sites but I don't know if they are still maintaining that. Jump to 1:00 mark from Unreal Engine GDC 2009 video;

CS:GO

http://blog.counter-strike.net/science/maps.html

[selmitto]

I was surprised that they created a single tool with many important features, instead of just the heat map.

Edited June 3, 2013 by Al Anselmo~Intelect0

[blackdog]

CUT

Yeah, I remember about Valve games, they publicly let people know in different occasions and games they do profile.

Thanks for the UE3 reminder, I usually watch all the tech demos and trailers, so I must have seen it and forgot about it.

[Kedhrin]

we had 2 forms of data collecting in Nexuiz, one was a heatmap generator that wasn't a heatmap per say, but it actually put actors in the level on an invisible layer that showed where a person killed someone/someone died - as well as a line showing their line of sight (blue for attacker, red for dead guy with a blue line between).

we also had the insane amount of stats pushed after every single match for nexuiz too (i think we push over 250 every match?).

the problem we ran into (and won't be something that is fixable because it'd require an entire rewrite) is the server queries... it messed up our database load pretty damn bad.

Another problem we had was, even though the data is super awesome to look at, we just didn't have the people to really process it and implement it - towards the end of Nexuiz i was the only designer on the project and just didn't have the time to focus on that information.

Well polished tools like this are super awesome. I've been eyeballing RAD's telemetry collection tool as well - but heatmaps, to me, only go so far. You can't tell gameplay 100% from a top down view - especially in an FPS.

[blackdog]

Thanks for the insight Kedhrin!

I guess the heatmap can help you evaluate if a map is really unbalanced. It was nice to see Valve applied the principle to sp also (at least, it was the first time i saw something like that).

Speaking of tools, I dunno if this is a totally stupid idea, but when I was testing an HL2 sp mod, i got the idea that in dev mode you could have a "debug gun" (which could simply be a console command), which would allow the tester to mark a location and enter a comment right away. On his end the level designer could scroll a list of entries, and the map editor would reposition the camera/select areas/entities accordingly.

Edited June 4, 2013 by blackdog

[e-freak]

Well polished tools like this are super awesome. I've been eyeballing RAD's telemetry collection tool as well - but heatmaps, to me, only go so far. You can't tell gameplay 100% from a top down view - especially in an FPS.

yeah especially in games like TF2 or any other assault game mode: more deaths towards the end of the map (and maybe around the spawn) should be normal. anyone playing dustbowl knows that you get loads of death on the final stretch, but its not a broken layout its just what's supposed to happen. Same can be translated to Deathmatch or anything if you supply powerful items in certain locations etc.

[blackdog]

yeah especially in games like TF2 or any other assault game mode: more deaths towards the end of the map (and maybe around the spawn) should be normal.

In fact that first example in the Dirty Bomb video seemed counter-intuitive to me, i was thinking what you just wrote. Of course you don't want in that gameplay mode to have your way paved and have an invalicable wall at the last objective, but then spreading very eavenly -like seems in the video- doesn't seem right neither. But that's really peculiar to the result you want to achieve.

Edited June 4, 2013 by blackdog

[BaRRaKID]

Information overload is the biggest problem with this kind of systems. You shouldn't profile every part of your game/system just for the sake of it, you need to plan it properly and look at profiling as a complement to real testing. If your testers (or you) complain about a certain part of the game or detect a problem you should profile that part, and only that part, during a certain period of time (this is another common mistake, usually these type of systems are on all the time, you don't need that, one week of data is as good as two weeks if you haven't made any changes in between). If there is a problem with map balance for example you don't need two months worth of data on how many kills per weapon you've on all maps, you just need to know for that specific map, where most of the battles occur and which team/player won the battle, basically who killed who and where. You can add weapon stats if there's a problem with a certain weapon being too advantageous at a certain location, but only if that's the problem you're trying to solve.

About database overload, for multiplayer games there is usually a server/computer that hosts each match, that host should be the only machine responsible for gathering the data, you don't really need to measure players (unless you want to have leaderboads and stuff like that off course) what you want to do is measure matches and that's the host job. Also you usually won't need real time data, what you want is to gather data during a certain period of time and analyze it later, so database updates should be done after the match ends, or once an hour or once a day if the host keeps running after each match is over. You can also invert it and keep the data saved on each host filesystem (or whatever) and then have your profiling sever ask the hosts for it when you need it. Basically instead of each host saying "Hey Profiler, I just finished a match here is what happened" have the profile say to each host "So, what have you been up to in the last few days?".

If you really need a system that does lots of queries just use a scalable cloud solution like Amazon or Rackspace.

[-HP-]

I know an awesome tool called focus testing.