3dsMax and VRay both have a tendency to crash when you exceed your system memory, that’s a long establish fact that frankly isn’t unique to rendering packages. But there are things you can do to reduce your memory usage, options buried deep within with cryptic names like “BSP Trees” and “Face/Level Coefficient”. If you want to know how to tweak these options to best effect on your rig, check out this great article from RenderStuff.
As we know, V-Ray uses the organization of the scene geometry in a form of a BSP tree to accelerate the raycasting process. It is logical to assume that the deeper our tree structure, the smaller its leaves, and the smaller the units of the geometry each leaf contains, then the more space in memory we need to hold tree’s branches and the greater is the potential raycasting speed it can provide.
However, there is a limit in such acceleration. After a certain threshold, further division is not reasonable, since the excessively branched structure of a BSP tree makes the work with itself too difficult. The raycasting algorithm may spend the computational resources for walking through the huge BSP tree, ruining all the gain in speed.
