One possible last thing to try for this card, actually. In addition to the "Restart in MS-DOS Mode" option you used, you can also boot more "pure"- this is done by pressing F5 as Windows is starting. (even before it shows the Windows 95 booting logo- I usually just go ham on the key as I boot the system) This should boot to plain MS-DOS. You'll be at a C:\> prompt. You'll want to use the cd command to change to and move to the directory containing the software in question and start it by typing the name of it's executable file. (dir can be used to view the contents of the current directory).
(I want to note that the above instructions are only there because you said you weren't fluent with MS-DOS so if it comes off as patronizing, I apologize!)
There is also a possibility that the software is expecting a very specific graphics card; say if the software was provided as part of an industrial package as part of the machine. For obvious reasons that could be problematic. I would expect that to be documented but it sounds like you've looked through the documentation and I expect something like that would have stuck out.
Just speaking from my own experience for cards in that "era", I recall that cards from S3 had good MS-DOS compatibility. S3 Trio64V and S3 Virge come to mind in particular.
I've seem glowing reviews about the MS-DOS compatibility of Matrox cards- but I've also heard the exact opposite. In the latter case, the individual was always talking about video games, so it's unclear if it would apply to a more industrial/business software program that uses graphics mode.
