The Communications Compatibility and Security Agreement (COMCASA) that was signed on Thursday had been stalled for years because of India's concerns that it would open up its communications network to the US military.
Sources said, United States sanctions on the import of Iranian crude oil and boosting cooperation in the Indo-Pacific region will also be the focus of the talks.
Ahead of his India visit, Pompeo also met new Pakistani Prime Minister Imran Khan on Wednesday, saying he was hopeful of "a reset of relations".
The drills would be a first of sorts - the two countries' forces have not previously trained simultaneously in the air, on the land and at sea, Reuters reported.
Mattis said the two sides also agreed to "increase and expand our engagement in the maritime domain" with a new joint exercise on India's coast in 2019, and a hotline between the two countries.
"That doesn't mean we can't be partners".
"I think that we're pretty much there already, on the American side", Mattis added.
"It's true both countries want to manage China, but to say that they want to contain China is not true", he said. Our effort here, too, is not to penalise great strategic partners like India, a major defence partner.
A senior USA defense official said the United States had only signed similar pacts with fewer than 30 other countries.
According to Wang, on anti-terrorism issues, India will continue to ask the United States to create pressure on Pakistan.
The world's two largest democracies have drawn closer in recent years, seeking ways to counter-balance China's spreading influence across Asia, notably in Pakistan, Southeast Asia and the Indian Ocean.
It will also allow the installation of high-security USA communication equipment on defence platforms being sourced from the US.
Over the last 10 years, since the nuclear deal came into force, India has bought $18-20 billion worth of defence equipment from America.
"There are very detailed conversations taking place between the U.S. and India on just the technical issues related to going to zero and those conversations will continue", the official added.
India's reluctance to disrupt its Iranian oil imports stems from mounting domestic pressure on the government of Prime Minister Narendra Modi because of rising oil prices and a weakening rupee against the dollar - months away from a general election. Khan said later he was optimistic he could reset the relationship with Washington after the US cut aid payments over Islamabad's alleged failure to combat the country's militants.
"Today, Secretary Pompeo and I bring the same spirit that Prime Minister Nehru carried to Washington nearly 70 years ago..." he added. "We have also discussed the H-1B visa issue". However, the US Congress, on August 1, granted President Donald Trump and Pompeo the power to waive sanctions.
India is Iran's top oil buyer after China, and it is seeking a waiver from the United States.
Instead, Delhi committed to buy Russian and even a French nuclear reactor because these are government-to-government contracts, while Westinghouse is a private U.S. company.