Hmm.. Interesting.

Well my original amp had both the amp and HU grounded to the negative terminal on the battery.

You guys are talking about "stereo battery ground" and "starting battery ground" but I only have one battery. Where should I ground the HU?

My input gains are maxed out so maybe thats the problem.