[ Midtown Madness 2 Central ] [ Midtown Madness 2 Central ]

Midtown Madness 2 Central > General Discussion > CMP - Car mesh packages
Goto page 1, 2  Next
View previous topic | View next topic
Author
Message Post new topicReply to topic
Yallis
Professional Sucker
Moderator

Joined: 20 Jul 2002
Posts: 693
Location: city.psdl

Status: Offline


Post subject: CMP - Car mesh packages Reply with quote


The CDDS thread derailed into CMP discussions, so here's a thread on another format that took some time to figure out. A rough understanding of the overall CMP structure was compiled shortly after the release of Midtown Madness 3, but a crucial part, the actual vertex data, made little sense. Years later I stumbled upon an OpenGL extension from NVIDIA for storing three floats in 32 bits. This caught my interest, and sure enough, the Xbox' NV2A supports a similar data type, D3DVSDT_NORMPACKED3. MM3 car vertex coordinates are 11/11/10-bit integers, scaled by the dimension of its axis-aligned bounding box in the vertex shader. This leads to bus bandwidth and RAM requirements reduced by 2/3 at the expense of a significant loss of precision. Probably a decent trade-off considering the big cities and the Xbox' small memory capacity (64MB).

The CMP format is used for player cars, ambient cars and metro trains. Every CMP file is called body.cmp and is placed in the Cars/carname(_lod1)(.N) folder of the game data archives. The _lod1 suffix is used for lower level of details on player cars. Ambient cars have LODs combined inside a single CMP file. The .N directory suffix is used for skins that require different texturing. Regular color change paint-jobs are achieved by using a single CMP file, but multiple separate material set files.

Instead of typing out a pseudo code interpretation of the entire file format that will be obsolete by tomorrow, I'll point to the C-like 010 Editor templates in my gameformats git repo. Pull requests accepted. Wink

File structure
CMP files are structured like a scene graph. Group nodes apply transformations and common properties to their children, and leaf nodes are used for rendering meshes, light and smoke sources. Judging by the name of nodes, the CMP files appears to have been assembled from various source files. While most of the format structure is now known, the meaning of many of the values remains unknown. When testing different values to observe the effects I have primarily used the car selection screen, because it's possible to change and reload files live on a debug enabled Xbox in a matter of seconds. But many properties of the files are perhaps only applicable when racing. Such properties could be damage deformations, weight and strength of loose parts, etc.

Root node (type 0)
Every CMP file starts with the root node. This is a group node with some special properties, including a format version identifier. There are three different versions of CMP files shipped with the retail game, each with slight differences in how the format is parsed. Besides the bounding box, matrices, material set path and a few counters, the majority of the properties are unknown. One of the unknown sections may be bound box line segments, but I haven't verified this theory.

Transformation node (type 1)
A basic group with render flags, transformation matrix, bound box and zero or more children. Ambient cars is a special case where the group may reference a mesh index, but not provide any children. Vertices with a matching mesh index found in the main multi-mesh node will be transformed below such groups.

Mesh node (type 2)
This is where the magic happens, as they would say on MTV Cribs. A player car mesh node can have up to two mesh sections; full detail and shadow mesh. Ambient cars use a special multi-mesh node instead.

Each mesh section contains a cheese-load of unknown values, but primarily a list of vertex indices, a list of vertices, a list of primitives and a list of materials.

The order of these lists is very important to how the mesh is rendered. The vertices are listed so that they can be referenced in ranges, making up triangle strips. A triangle strip primitive entry is basically a list of arguments to D3D's DrawPrimitives(). Degenerate strips are used extensively, so a car may consist of just a few consecutive strips. This causes wire-frame renderings to look rather funky.

The index list consists of indices in the vertex list, making up separate triangles, also referenced in ranges. This fills the holes not covered by the triangle strips. A triangle list primitive contains the arguments to D3D's DrawIndexedPrimitives().

After the list of primitives the primitives are repeated, but with an additional value; an index in the external material set. Based on the material type the material's color, texture or the per-vertex colors are applied to the primitive.

A vertex section is 24 bytes long and starts with the position coordinate, normal vector and UV coordinate, all in the D3DVSDT_NORMPACKED3 11/11/10-bit format. The third value of the UV coordinate is unknown. So is the next 10 bits. The following 8 bits is a per-vertex color in the form of a material index multiplied by 11. It sounds silly, I have probably missed something here. After another 5 unknown bits is a (probably) 3 bit mesh id. The mesh id is used by ambient cars to combine all the car's vertices in a single buffer. The remaining 80 bits are unknown. The last part appears to control shininess in some way.

The mesh section ends with an optional list of license plate vertices, probably used to map the dynamic user-selectable 6-letter string from the license plate texture. Haven't investigated this further.

An empty mesh section can reference another mesh node by it's name.

Axis node (type 3)
A special node with no content. My theory is that these nodes are used to flag their parent transformation node's matrix as an attachment point for loose parts. For example a hood would be attached where the hinges are located until it's fully smashed off. Haven't verified this.

Light node (type 4)
Lights. Attributes include type, color and size. Some unknown values. In-game testing indicates that some of the values affects blink frequency/siren rotation.

Smoke node (type 5)
Presumably a smoke source. Only a single value, probably type, thickness or color. Who knows.

Multi-mesh node (type 6)
Used by ambient cars. Like the regular type 2 mesh, but includes three mesh sections; full, LOD and shadow. All of the car's mesh is combined in this node and then referenced by other transformation nodes on a per-vertex level by a mesh id.

Related files
.omb - Simple material list. Name, texture, color, type.
.ccol - Simple collision mesh. List of triangles with normals and some other property.

There's a lot of small details that I've probably forgot documenting both here and in the binary templates. I expect some of the remaining unknowns to include shininess/specular color and possibly some mesh deformation properties and secondary texture coordinates for damage.

Example of how mesh nodes are arranged

Axis-aligned bound box hierarchy.


Transformed mesh bounds.


Wheels. This model has two different wheel mesh nodes, the remaining 4 nodes are empty references to the earlier meshes.


Interior and windows (two separate nodes).


Car body.


Loose parts (each part is a separate node).


Low-poly shadow mesh.


Wireframe of all mesh nodes.


All mesh nodes, except shadow.


And here's a summary of the nodes in the file: 2015-01-10-MM3_CMP_Cement_nodes.txt








Questions?
_________________


Last edited by Yallis on 12 Jan 2015 07:49 am; edited 1 time in total
Post 11 Jan 2015 09:34 pm
View user's profile Send private message Visit poster's website
Yallis
Professional Sucker
Moderator

Joined: 20 Jul 2002
Posts: 693
Location: city.psdl

Status: Offline


Post subject: Reply with quote

Some interesting stuff I noticed when dissecting the files:

The mv_BadGuys car used by some opponents in the game's missions is a regular player vehicle. It can be made selectable by adding a menu entry. Here's the police skin:


Some cars have surprisingly many details in places that's almost impossible to get a look at with the game's low resolution and fixed camera angles.


The Avalanche (named Asp in the game frontend, maybe some lawyers insisted?) is excellent for debugging a parser due to its low number of vertices, primitives and textures. The car is taken from an earlier DICE game, Motorhead, known for its ridiculous efficient and smokin' hot software rendering engine.

Another gem that is a little harder to spot is the hood emblem on the DLC car Furious. It's the old DICE logo:

But it's not like DICE have any shame when it comes to self-promotion.

And if you can't get enough DICE logos, check out their RalliSport Challenge games...

The logos on the unlicenced DLC cars Campione and Concept are the same. Not sure what this is a reference to, but if there's any doubt about what inspired the Campione, its texture name is lamborghinirip.cdds...


Another odd logo/name is this DLC car. It is named Carica in the frontend, but the badge says Charge.


Burken, Swedish for "the box".


Including a Saab was obligatory for the Swedes at DICE, but it struck me that there's quite a few skins with blue and yellow.


The Tre Kronor Hummer takes the cake...


I've been wondering if the FLE could be inspired by the Norwegian-made Ford TH!NK. That would explain the rosemaling skin.



_________________


Last edited by Yallis on 12 Jan 2015 07:50 am; edited 1 time in total
Post 11 Jan 2015 09:44 pm
View user's profile Send private message Visit poster's website
Yallis
Professional Sucker
Moderator

Joined: 20 Jul 2002
Posts: 693
Location: city.psdl

Status: Offline


Post subject: Reply with quote

Some screenshots from my test folder:

Ambient car.


Scaled up wheel mesh, disabled the remaining nodes.


Locating the per-vertex material ids took a while.


Disco police.

_________________
Post 11 Jan 2015 09:46 pm
View user's profile Send private message Visit poster's website
_Tim_
Kiwilicious
Administrator

Joined: 22 Mar 2005
Posts: 1259
Location: Where the wild things are.

Status: Offline


Xbox Live Gamertag:
TimNZ
Post subject: Reply with quote

You, are, amazing. Shocked Shocked Mr. Green Cool

Incredible work and an amazing read!
_________________
<- My experimental pages.
Post 12 Jan 2015 04:06 am
View user's profile Send private message
carking1996
MM2C Too Much

Joined: 12 Aug 2009
Posts: 608
Location: United States

Status: Offline


Post subject: Reply with quote

Wow!!
_________________
Post 16 Jan 2015 08:26 pm
View user's profile Send private message Send e-mail MSN Messenger
carking1996
MM2C Too Much

Joined: 12 Aug 2009
Posts: 608
Location: United States

Status: Offline


Post subject: Reply with quote

Also, any chances to get the extractor or well converter to see them?
_________________
Post 16 Jan 2015 08:28 pm
View user's profile Send private message Send e-mail MSN Messenger
aaro4130
MM2C Maniac

Joined: 22 Feb 2009
Posts: 234
Location: At my laptop (hooked up to a monitor, keyboard, mouse, and speakers)

Status: Offline


Post subject: Reply with quote

Absolutely amazing work!
Post 16 Jan 2015 08:35 pm
View user's profile Send private message Yahoo Messenger MSN Messenger
Yallis
Professional Sucker
Moderator

Joined: 20 Jul 2002
Posts: 693
Location: city.psdl

Status: Offline


Post subject: Reply with quote

Thanks, fellas!

carking1996 wrote:
Also, any chances to get the extractor or well converter to see them?

The code for the experimental parser and viewer is available in my gameformats repo as well. The viewer uses the OpenGL abstraction library OpenSceneGraph, which in turn has a cascade of other dependencies, so it's probably a nightmare to compile for Windows. I have planned to make a CMP <-> COLLADA conversion utility for universal availability, but I want a more complete understanding of the CMP format first. Feel free to further hack on the code if you're interested.
_________________
Post 17 Jan 2015 05:34 pm
View user's profile Send private message Visit poster's website
carking1996
MM2C Too Much

Joined: 12 Aug 2009
Posts: 608
Location: United States

Status: Offline


Post subject: Reply with quote

Excellent!
_________________
Post 27 Jan 2015 12:56 am
View user's profile Send private message Send e-mail MSN Messenger
aaro4130
MM2C Maniac

Joined: 22 Feb 2009
Posts: 234
Location: At my laptop (hooked up to a monitor, keyboard, mouse, and speakers)

Status: Offline


Post subject: Reply with quote

Compiled on Windows but crashes instantly after "Reading xxx.cmp". I had to hardcode the full CMP path in, then it crashed at this:
Post 27 Jan 2015 02:13 am
View user's profile Send private message Yahoo Messenger MSN Messenger
Yallis
Professional Sucker
Moderator

Joined: 20 Jul 2002
Posts: 693
Location: city.psdl

Status: Offline


Post subject: Reply with quote

Identified the remaining vertex attributes. This includes damage texture, mesh deformation, per-vertex lighting properties (shininess and reflection) and some ids (damage group, matrix id). Code and binary template updated in the git repo. The number of unknown values in decreasing. Smile



This is my current interpretation of the vertex structure:
Code:
struct Vertex
{
    // Scaled position, D3DVSDT_NORMPACKED3
    int   x  : 11;      // x  / 1024 * scale
    int   y  : 11;      // y  / 1024 * scale
    int   z  : 10;      // z  /  512 * scale
    // Normal, D3DVSDT_NORMPACKED3
    int   nx : 11;      // nx / 1024
    int   ny : 11;      // ny / 1024
    int   nz : 10;      // nz /  512
    // UV0: main tex, UV1: demolition tex, D3DVSDT_NORMPACKED3
    int   u0 : 11;      // u0 / 1024
    int   v0 : 11;      // v0 / 1024
    int   u1 : 10;      // u1 /  512
    // Ids divided by max value to prevent overrun in shader
    ubyte materialId;   // id /   11
    ubyte matrixId;     // id /   37
    ubyte v1;           // v1 /  256
    ubyte demolitionId; // id /   23
    // Deformation position offsets, D3DVSDT_NORMPACKED3
    int   dx : 11;      // dx / 1024
    int   dy : 11;      // dy / 1024
    int   dz : 10;      // dz /  512
    // Lighting and reflection
    ubyte specularPower;
    ubyte envmapIntensity;
    ubyte ambientIntensity;
    ubyte specularIntensity;
}



aaro4130 wrote:
Compiled on Windows but crashes instantly after "Reading xxx.cmp". I had to hardcode the full CMP path in, then it crashed at this:
...

The program takes the first command line argument as the path to the CMP file. This shouldn't be treated any different from what you have hardcoded. It's difficult to guess what's causing the segfault without any context. Is ifs an open, readable stream? By enabling exceptions on the stream you can catch read errors:
Code:
ifs.exceptions(std::ifstream::failbit | std::ifstream::badbit | std::ifstream::eofbit);

_________________
Post 08 Feb 2015 11:26 pm
View user's profile Send private message Visit poster's website
aaro4130
MM2C Maniac

Joined: 22 Feb 2009
Posts: 234
Location: At my laptop (hooked up to a monitor, keyboard, mouse, and speakers)

Status: Offline


Post subject: Reply with quote

adding the exceptions did nothing. Same error. :/
Post 09 Feb 2015 12:22 am
View user's profile Send private message Yahoo Messenger MSN Messenger
Yallis
Professional Sucker
Moderator

Joined: 20 Jul 2002
Posts: 693
Location: city.psdl

Status: Offline


Post subject: Reply with quote

Implemented per-vertex transformations to verify the multi-mesh nodes used by ambient vehicles.


The most interesting ambient CMP is this unused, unfinished bus. I guess they gave up on the AI for long vehicles.


aaro4130 wrote:
adding the exceptions did nothing. Same error. :/

Are you positive you're not doing something funky? Tried compiling with VS2013 and it appears to work:


_________________
Post 13 Feb 2015 11:31 pm
View user's profile Send private message Visit poster's website
aaro4130
MM2C Maniac

Joined: 22 Feb 2009
Posts: 234
Location: At my laptop (hooked up to a monitor, keyboard, mouse, and speakers)

Status: Offline


Post subject: Reply with quote

Yeah I am. It's weird how ifstream wouldn't work from an argument. You should just post your build at this point :p
Post 14 Feb 2015 12:19 am
View user's profile Send private message Yahoo Messenger MSN Messenger
Yallis
Professional Sucker
Moderator

Joined: 20 Jul 2002
Posts: 693
Location: city.psdl

Status: Offline


Post subject: Reply with quote

aaro4130 wrote:
You should just post your build at this point :p

I compiled all the deps for Win64, so it isn't very portable. It would be obsolete by the next commit anyway. Here's a minimal VS2013 solution and executable demonstrating the parser: cmpparser-2015-02-15.7z
_________________
Post 15 Feb 2015 10:11 pm
View user's profile Send private message Visit poster's website
Display posts from previous: Post new topicReply to topic

Page 1 of 2 All times are GMT
Goto page 1, 2  Next


Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Home - MM2C.com - Contact - Staff & Seniors - FAQ - Community Rules - Syndication


Powered by phpBB © 2001, 2005 phpBB Group