It's a mix of garage rock, blues and free jazz. Though Captain Beefheart tossed off a couple of stabs at commercial rock in the mid-70s (unheard by me, though they're supposedly bad). Trout Mask Replica—77 minutes of atonal insanity (Beefheart mentor Zappa looks tame in comparison)—is his most well-known album, though you're just as likely to hate it as you would love it. His first album, Safe as Milk is a solid set of slightly off-kilter, though fairly accessible blues rock.
My favorite album of his, however, might be Lick My Decals Off, Baby, which was released after the Zappa-produced TMR. It's a lot like that album, though a little less abrasive and about half the length. Import vinyl (a copy of which I have) or a file-sharing service is the only way to go with this one; it was only in print on CD for like a day, and that was several years ago.