Speaking at the annual Shangri-La Dialogue in Singapore, Mattis said China's placement of weapons systems, including missile batteries and military aircraft, at outposts in the disputed sea raised questions about Beijing's intentions.
He warned that America's recent move to disinvite China from a multinational naval exercise this summer was an "initial response" to the militarization of the islands. Mattis called the US action a "relatively small effect". He said he would visit Beijing soon at China's invitation.
"I believe there are much larger consequences in the future when nations lose the rapport of their neighbours. eventually these (actions) do not pay off", Mattis said.
Last week, the US withdrew an invitation for Beijing's participation in naval exercises known as the Rim of the Pacific.
"There are consequences that will continue to come home to roost, so to speak, with China, if they don't find a way to work more collaboratively with all of the nations who have interests", he said. It says military activities on the islands have been for self-defense purposes.
Taiwan and China, which are hundreds of miles to the north, also claim some of the islands. China claims Taiwan as its own and it is one of its most sensitive issues.
Mattis's address in Singapore returned to a theme that he and other senior USA officials have hammered home since Trump took office - that America is here to stay in the Asia-Pacific region and that allies should stick with Washington instead of aligning with Beijing.
Gen Mattis said the issue of removing U.S. troops from South Korea was "not on the table" and that "our objective remains the complete, verifiable, and irreversible denuclearisation of the Korean peninsula".
The U.S. defense secretary replied that freedom of navigation operations were not militarizing the waters.
Mattis told Asia's biggest security conference Saturday that China's deployment of missile batteries and bombers to outposts in the disputed area was "a reality" the USA and its allies would have to deal with.
He's comments were the first at such a public and worldwide event to acknowledge Beijing's plans to base both troops and weapons on its natural and man-made islands in the Paracel and Spratly archipelagoes. Trump announced Friday that the meeting was back on. The summit is expected to be here in Singapore.
The previous week the US had withdrawn an invitation for Beijing to participate in the exercise known as Rim of the Pacific. China had participated in the exercise known as Rim of the Pacific in 2014 and 2016.
China has also landed heavy bombers on Woody Island in the Paracel Islands.
Earlier US Defence Secretary James Mattis said Beijing's actions called into question its broader goals.
Last Sunday, Beijing protested what it called a "provocation" after United States warships came within 12 nautical miles (around 22km) of the disputed Paracel Islands, among a string of islets, reefs and shoals over which China has territorial disputes with its neighbours. It was the latest freedom of navigation operation created to challenge Beijing's claims. China protested the maneuver.
In recent years, the US had sought to stabilize military relations with China, but the militarization of the islands has been a persistent point of conflict.
"Yet China's policy in the South China Sea stands in stark contrast to the openness our strategy promotes".
China's delegation at the conference is being led by Lt. Gen.
Meanwhile, He, who is also vice-president of the PLA Academy of Military Science, criticised Mattis' remarks regarding Washington's relations with Taipei.
He responded by saying: "The Chinese government and its people strongly oppose arms sale to Taiwan, all kinds of official government contacts to strengthen their ties with Taipei, as well as other acts to harm the "one-China principle".
"The US will continue to pursue a constructive, results-oriented relationship with China, cooperating when possible and competing vigorously where we must. of course we recognise any sustainable Indo-Pacific order has a role for China", he said.