I have a 2008 OBV with the following setup regarding ballast:
- 750# Rears
- 400# Floor (ski locker)
- 750# IBS
All this discussion below assumes you have a similar system and not 3 individual reversible pumps! I don't know when they did a hard switch to reversibles, however, if it takes as long as you say it does to fill I don't believe you have reversibles. Easy check if you do, does your ballast drain above the waterline?
I have the original Gravity III system (if you have more than 1 ballast switch, then you have the GIII - typically the 3 independent switches are located to the right, under the throttle). The original system had one massive fill pump, aerator style, usually around 1,200 gph (possibly higher if you have a beast of a pump) that fed a distribution manifold of sprinkler valves. Each ballast switch was redundant for turning the pump on, but controlled each sprinkler valve solenoid independently. The problem with these valves is they rely on water pressure to open the diaphragm - the supply pump only puts out about 10 PSI - far less than your house supply at 60+ PSI for watering your lawn. The drain pumps are typically smaller (6-800 gph) aerator pumps located close to the bottom of the bags. In all my seasons, I can think of one time I had to wait on ballast to drain and that's when we were getting rained on suddenly and were unexpectedly pulling the boat out - and that said, if I had started draining them when I should have it would not have been a problem.
I did some in-depth analysis before I spent $100+ in valving that some said would not work well. I compared the flowrate across a generic sprinkler valve vs a streamlined sprinkler valve (not these servo valves I ended up finding). The results were impressive and highlighted the weakness in the original GIII valve system. Most all of your flow losses are found across the valve! Now, bear in mind, this model assumes a general loss due to tube diameter and similar connector size, however, only assumes roughly 10' of line from the manifold to the bags, i.e. 3/4" ribbed hose vs 1" smooth hose, 90* fittings, etc. Take it for what it's worth, it's just math:

This shows you gain significant time reduction by replacing all your sprinkler valves to the servo valves, roughly 20%. If you change to 1" hose and fittings, you'll save another 15%. Again, just math and pump curves with losses. Individual setups will vary greatly so don't quote me on X number of minutes per $ spent. That said, my half 3/4" half 1" system impresses me.
Here are the old valves in the manifold setup:

It used to take me close to 30 minutes to fill all my ballast (minus the IBS bag, she's new as of last year AFTER the valve upgrade). I can now fill 100% of my ballast between the time we launch and our riding spot (15-20 mins tops) using the same stock aerator pump.
I purchased 3 of these 1" servo style ball valves. You have to be specific when ordering to make sure the wiring is setup for power open, no power spring close so you don't have to develop compound relays to handle the power off closure. They were roughly $40 each. I could dig up the supplier and part number if anyone is interested. I designed the new manifold to have a 2nd supply source and was planning to add a 2nd Rule pump to double the volume, however, after 2 months of practice I have decided there are other more needy uses of thru hulls in my future (vents and new drains for IBS). The capped off leg of the new manifold will house a 4th servo valve and will control the fill of my surf side bag that is located under the port side seating where the cooler used to be.
New manifold with servo valves:

My suggestion is to invest in better valving, free-er flowing fittings and hose and if you must have more than 1 fill pump, have 1, 2 or 3 aerators feeding a controllable manifold. The power amperage will remain low so you won't have to rewire everything to 8 gauge and you don't have to worry about timers and running them dry. Plus they do move more water than reversibles. The only problem with aerators is you run the risk of them air locking which can be a PITA on just one feed pump, let alone multiples. You think your fill times are bad now, wait until you hit the fill button and wait 20+ minutes while you motor over to your spot only to find out a pump hasn't been doing anything but spinning air! 
I have read a few posts about tapping into the RAW water pump supply to fill ballast. My thoughts and just an opinion are this:
- If this was a significant advantage, there would be kits and/or new boats coming like this from the factory. The only other space age idea of rapid fill ballast is the RAMFILL design by Centurion, but that requires a dedicated hull design and system.
- Anytime you pull water from cooling the motor, this seems like a really bad idea. Even if it takes a record low 10 minutes to fill your ballast, you would be starving your (hopefully idling) engine of much needed coolant during that period - even worse if you're up on plane trying to get to your favorite spot while not cooling your engine.
2008 OBV
325 EFI with closed cooling
Gravity III with 750's in the rear, 400 stock floor, 750 IBS
Servo valve ballast upgrade
Wetsounds Rev10
JL Amps, WS420BT
2000 Outback LS <-- Sold!!!
310 Carb Direct Drive
800# Rear Locker
750# IBS (should have kept this for the OBV)
Dual Batteries with 2 Bank Charger System
Kenwood Head with Remotes
Too many hours restoring exterior from prior negligence to count!