NOPE, Rodeo. It can not be done. Silly girl.
Always asking for the moon!
Wait, hmmmmm....sure, now that i think about it.....
PHP post maybe. Since the form page already KNOWS the category NUMBER by the time it loads the Custom form then the form div could load with a specified name or simply the Category number written dynamically into the ID. Even class names inside the form ID could utilize the Category number added into the class name as a prepend or append value to immediately refer to special form specs you already have written in the CSS file.
In fact, now that I am thinking...there could be an Admin panel --a text area field where you drop in the modified CSS for the specific Category form of any category. Save and see that Category form respond "live".
It would take jiggering with the core code at the moment.
I think there are probably several other elegant ways to do this for programmers who really know php....or even do-able with jquery.