I made very small modifications to the arduino-tiny core, specifically 4 lines of preprocessor directives to disable code. I am commenting out code in both HardwareSerial.cpp and HardwareSerial.h in order to have VGATonic handle the UART interrupts on its own (this is the relevant code block that interferes):

ISR(USART_RX_vect) 
{
  uint8_t tmp = UDR;
  if (rxn==BUFFER_SIZE)
    rxn=0;
  RX_BUFF[rxn++] = tmp;  
  rx_flag=1;
}

A strict reading of license terms counts disabling code as a modification, so I wanted to document that arduino-tiny is licensed under LPGL. I have released my modifications to arduino-tiny on Github here.

You can directly overwrite the arduino-tiny files when installing arduino-tiny core, but on a Mac they go here (Arduino IDE 1.6.5):

Arduino (Right Click) -> Show package contents
/Contents/Resources/Java/hardware/tiny/avr/cores/tiny/