Australia will build eight nuclear-powered submarines under an Indo-Pacific security partnership that has riled China.
"It will provoke other powers to also act more aggressively in the region, especially in the South China Sea," Malaysia's Prime Minister Office said in a statement.
The statement did not mention China, but Beijing's foreign policy in the region has been increasingly assertive, particularly its maritime claims in the resource-rich South China Sea, some of which conflict with Malaysia's own claims.
"As a country within ASEAN, Malaysia holds the principle of maintaining ASEAN as a Zone of Peace, Freedom and Neutrality (ZOFPAN)," the statement said.
Malaysia urged all parties to avoid any provocation and arms competition in the region.
READ MORE: Australia says it was 'upfront' with France over submarine deal as crisis continues