Well, I got a similar problem, but I know what causes the problem. My driver has difficulties rendering fog. On the etpub server r_wolffog is forced to 0, so it's no issue there, but for example on the silent server I get terrible fps. Your driver probably has difficulties rendering some graphics that are shown around players, or the players themselves (good fps with low amount of players, since cpu can tak over render calculations, but not for high amount of players since cpu is a lot slower). Only solution is to change vid drivers or get a new gfx card.