Stocks traded slightly lower throughout the morning, but took off immediately after the Fed's decision in the early afternoon. Bond yields fell sharply — their biggest move in nearly two years. The price of gold had its biggest one-day jump in four years as traders anticipated that the Fed's decision might cause inflation.
Fed policymakers decided to maintain the central bank's $85 billion in monthly bond purchases, a program that has been in place since December 2012. The bond purchases encouraged borrowing by keeping interest rates low and encouraging investors to buy stocks by making bonds more expensive in comparison.
While the U.S. economy appeared to be improving, the bank's policymakers "decided to await more evidence that progress will be sustained" before deciding to slow the bond purchases. The bank also cut its full-year economic outlook for this year and next.
Stock traders shrugged off the Fed's dimmer outlook and focused on the prospect of continued stimulus.
The S&P 500 surged 20.76 points, or 1.2 percent, to 1,725.52, slicing through its previous all-time high of 1,709.67 set on Aug. 2.
The Dow Jones industrial average jumped 147.21 points, or 1 percent, to 15,676.94, also above its previous record high of 15,658.36 from Aug. 2.
The Nasdaq composite rose 37.94 points, 1 percent, to 3,783.64.
The fate of the Fed's economic stimulus program has been the biggest question on Wall Street for months. It was widely expected that the Fed would cut back on its bond buying at the September meeting.
Tom di Galoma, a bond trader at ED&F Man Capital, said he was "completely shocked" that the Fed decided to wait.
Some investors advised caution, even as the stock market hit all-time highs.
While the Fed's decision is positive for the market in the short term, "investors need to take a step back and consider the idea that maybe the U.S economy is on weaker footing than we originally thought," said Marc Doss, regional chief investment officer for Wells Fargo Private Bank.
Bond prices also rose sharply, sending yields lower. The yield on the 10-year Treasury note fell to 2.68 percent from 2.87 percent a minute before the Fed released its statement. It was a rush into bonds not seen since October 2011. The yield on the 10-year Treasury note is a benchmark for many kinds of lending rates, including home mortgages.
Investors said there were two reasons why bond prices jumped Wednesday. First, since the Fed decided to keep up its bond purchase program, it keeps a big buyer of bonds in the market. Also, the Fed's lowered economic outlook gave traders another reason to seek long-term safety in their portfolios.
As bond yields plunged, investors snapped up stocks that tend to pay richer dividends, such as utilities. The Dow Jones utility average jumped 3 percent, its best day in two years.
Stocks of home builders also rose as investors speculated that the Fed's pledge to keep interest rates low would continue to benefit the housing market. Pulte Homes, Hovnanian and Toll Brothers were up more than 5 percent each, while D.R. Horton jumped nearly 7 percent.
The price of gold jumped $55, or 4 percent, to $1,364 an ounce.
In June, Fed Chairman Ben Bernanke laid out a possible timetable for easing up on the bond purchases, and pledged to end them by the middle of 2014, if the economy continued to improve.
The Fed's next meeting is October 29-30, another opportunity for the central bank to start reducing the program.
Wells Fargo's Doss and other investors said the Fed might be waiting to see what happens in Washington, D.C. in the coming weeks. A debate over the debt ceiling and a showdown between Congressional Republicans and the White House over the budget looms.
Bernanke probably kept the stimulus in place because he wanted to be certain the economy was ready to function without the Fed's help, said Matt Tom, head of public fixed income at ING U.S. Investment Management.
Cutting back before the economy was ready would have been much more destabilizing to the market, he said.