USB audio connectivity has recently been added, based partly on earlier work by @Michele Perla

You can now place AudioInputUSB and AudioOutputUSB objects in your Arduino sketch and connect them to any of the other Teensy Audio Library stuff. Any sound send to a AudioOutputUSB object arrives on your PC/Mac as if you'd connected a line-in. Sound your PC plays (if the Teensy's audio is selected in your sound control panel) arrives at the AudioInputUSB object.

Currently this is all very new code, only on Github. Details are here:

https://forum.pjrc.com/threads/24309-USB-Audio-for-Teensy-3-0/page2

Within the next few weeks, a Teensyduino beta test installer will be published, to make trying this much simpler. But if you'd like to give this a try, of course you can download the stuff from github and copy to the correct location within your copy of Arduino, and edit boards.txt to make Audio appear in the Tools > USB Type menu.

At this very early stage, the design tool hasn't been updated with these new objects. You'll need to edit the audio objects right in your sketch. The simplest way involves using the I2S objects and then you just change from I2S to USB.