Oil prices fell for a fourth session after data showed an unexpected increase in crude and gasoline stockpiles. The S&P 500 energy sector notched a four-day decline of 4 percent, its weakest such period in 14 months.
"Oil coming off recent highs and as crude prices move so (do) the big energy stocks," said Peter Jankovskis, co-chief investment officer at OakBrook Investments LLC in Lisle, Illinois.
Exxon fell 1.3 percent to $81.21 and Schlumberger dropped 2.0 percent to $61.55 after touching $61.11, its lowest since January 2016.
Brent and US crude both fell after touching last week their highest in almost 2-1/2 years.
The tech sector was the largest weight on the S&P 500, something Jankovskis attributed partly to their "very strong run this year; perhaps some people are taking profits."
The Dow Jones Industrial Average fell 138.19 points, or 0.59 percent, to close at 23,271.28, the S&P 500 lost 14.25 points, or 0.55 percent, to 2,564.62 and the Nasdaq Composite dropped 31.66 points, or 0.47 percent, to 6,706.21.
Republican US Senator Ron Johnson said he opposes his party's Senate tax revamp proposal, the Wall Street Journal reported, leaving the passage of the proposal in limbo as the GOP holds a slim majority. Analysts have said the slashing of the corporate tax to 20 percent from its current 35 percent would likely be a boon for the stock market.
The CBOE Volatility index, a widely followed measure of market anxiety, hit a more than 2-month high at 14.51 and was last up 1.5 points at 13.13.
A rise in both inflation and retail sales sent a signal to the Federal Reserve, which had been concerned about a recent disinflationary trend, setting the US central bank on a path to raise benchmark interest rates in December.
Among the few Wall Street gainers on Wednesday were financial stocks, which rose on prospects of further rate hikes. The S&P 500 bank index added 0.61 percent.
High-yielding sectors like utilities and consumer staples, among the so-called bond proxies, were the largest decliners outside of energy.
Target shares tumbled 9.9 percent to $54.16 after it issued a disappointing profit forecast for the key holiday quarter.
After the closing bell, Cisco shares rose 3.4 percent after the company reported a 3.1 percent rise in quarterly profit driven by growth in its newer business areas.
Declining issues outnumbered advancing ones on the NYSE by a 1.78-to-1 ratio; on Nasdaq, a 1.60-to-1 ratio favored decliners.
The S&P 500 posted 35 new 52-week highs and 19 new lows; the Nasdaq Composite recorded 45 new highs and 84 new lows.
About 6.55 billion shares changed hands in US exchanges, compared with the 6.78 billion daily average over the last 20 sessions.